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Summary 



Internet services such as voice over Internet protocol (VoIP) and audio/video stream- 
ing (e.g. video on demand and video conferencing) are becoming more and more 
popular with the recent spread of broadband networks. These kind of "real-time" ser- 
vices often demand low delay, high bandwidth and low packet-loss rates in order to 
deUver tolerable quality for the end users. However, the heterogeneous communica- 
tion infrastructure of today's packet- switched networks does not provide a guaranteed 
performance in terms of bandwidth or delay and therefore the desired quality of ser- 
vice is generally not achieved. 

To achieve a certain degree of robustness on errorprone channels one can make use 
of multiple-description (MD) coding, which is a discipline that recently has received a 
lot of attention. The MD problem is basically a joint source-channel coding problem. 
It is about (lossy) encoding of information for transmission over an unreliable K- 
channel communication system. The channels may break down resulting in erasures 
and a loss of information at the receiving side. Which of the 2^ — 1 non-trivial subsets 
of the K channels that are working is assumed known at the receiving side but not at 
the encoder. The problem is then to design an MD scheme which, for given channel 
rates (or a given sum rate), minimizes the distortions due to reconstruction of the 
source using information from any subsets of the channels. 

While this thesis focuses mainly on the information theoretic aspects of MD cod- 
ing, we will also show how the proposed MD coding scheme can be used to construct 
a perceptually robust audio coder suitable for audio streaming on packet- switched 
networks. 

We attack the MD problem from a source coding point of view and consider the 
general case involving K descriptions. We make extensive use of lattice vector quanti- 
zation (LVQ) theory, which turns out to be instrumental in the sense that the proposed 
MD-LVQ scheme serves as a bridge between theory and practice. In asymptotic cases 
of high resolution and large lattice vector quantizer dimension, we show that the best 
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known information theoretic rate-distortion MD bounds can be achieved, whereas in 
non-asymptotic cases of finite-dimensional lattice vector quantizers (but still under 
high resolution assumptions) we construct practical MD-LVQ schemes, which are 
comparable and often superior to existing state-of-the-art schemes. 

In the two-channel symmetric case it has previously been established that the side 
descriptions of an MD-LVQ scheme admit side distortions, which (at high resolution 
conditions) are identical to that of i-dimensional quantizers having spherical Voronoi 
cells. In this case we say that the side quantizers achieve the L-sphere bound. Such a 
result has not been estabUshed for the two-channel asymmetric case before. However, 
the proposed MD-LVQ scheme is able to achieve the L-sphere bound for two descrip- 
tions, at high resolution conditions, in both the symmetric and asymmetric cases. 

The proposed MD-LVQ scheme appears to be among the first schemes in the liter- 
ature that achieves the largest known high-resolution three-channel MD region in the 
quadratic Gaussian case. While optimality is only proven for ii" < 3 descriptions we 
conjecture it to be optimal for any K descriptions. 

We present closed-form expressions for the rate and distortion performance for 
general smooth stationary sources and squared error distortion criterion and at high 
resolution conditions (also for finite-dimensional lattice vector quantizers). It is shown 
that the side distortions in the three-channel case is expressed through the dimension- 
less normaUzed second moment of an L-sphere independent of the type of lattice used 
for the side quantizers. This is in line with previous results for the two-description 
case. 

The rate loss when using finite-dimensional lattice vector quantizers is lattice in- 
dependent and given by the rate loss of an L-sphere and an additional term describing 
the ratio of two dimensionless expansion factors. The overall rate loss is shown to be 
superior to existing three-channel schemes. 
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Chapter J 

Introduction 



1.1 Motivation 

Internet services such as voice over Internet protocol (VoIP) and audio/video stream- 
ing (e.g. video on demand and video conferencing) are becoming more and more 
popular with the recent spread of broadband networks. These kinds of "real-time" 
services often demand low delay, high bandwidth and low packet-loss rates in order to 
dehver tolerable quality for the end users. However, the heterogeneous communica- 
tion infrastructure of today's packet- switched networks does not provide a guaranteed 
performance in terms of bandwidth or delay and therefore the desired quality of ser- 
vice is (at least in the authors experience) generally not achieved. 

Clearly, many consumers enjoy the Internet telephony services provided for free 
through e.g. Skype™. This trend seems to be steadily growing, and more and more 
people are replacing their traditional landline phones with VoIP compatible systems. 
On the wireless side it is likely that cell phones soon are to be replaced by VoIP 
compatible wireless (mobile) phones. A driving impetus is consumer demand for 
cheaper calls, which sometimes may compromise quaUty. 

The structure of packet-switched networks makes it possible to exploit diversity 
in order to achieve robustness towards delay and packet losses and thereby improve 
the quality of existing VoIP services. For example, at the cost of increased bandwidth 
(or bit rate), every packet may be dupUcated and transmitted over two different paths 
(or channels) throughout the network. If one of the channels fails, there will be no 
reduction in quality at the receiving side. Thus, we have a great degree of robustness. 
On the other hand, if none of the channels fail so that both packets are received, there 
will be no improvement in quality over that of using a single packet. Hence, robustness 
via diversity comes with a price. 

However, if we can tolerate a small quality degradation on reception of a single 
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packet, we can reduce the bit rates of the individual packets, while still maintaining 
the good quality on reception of both packets by making sure that the two packets 
improve upon each other This idea of trading off bit rate vs. quality between a number 
of packets (or descriptions) is usually referred to as the multiple-description (MD) 
problem and is the topic of this thesis. 

While this thesis focuses mainly on the information theoretic aspects of MD cod- 
ing, we will also show how the proposed MD coding scheme can be used to construct 
a perceptually robust audio coder suitable for audio streaming on packet-switched 
networks. To the best of the author's knowledge the use of MD coding in current 
state-of-the-art VoIP systems or audio streaming applications is virtually non-existent. 
We expect, however, that future schemes will employ MD coding to achieve a certain 
degree of robustness towards packet losses. The research presented in this thesis is a 
step in that direction. 

1.2 Introduction to MD Lattice Vector Quantization 

The MD problem is basically a joint source-channel coding problem. It is about 
(lossy) encoding of information for transmission over an unreliable A'-channel com- 
munication system. The channels may break down resulting in erasures and a loss of 
information at the receiving side. Which of the 2^ — 1 non-trivial subsets of the K 
channels that are working is assumed known at the receiving side but not at the en- 
coder The problem is then to design an MD scheme which, for given channel rates (or 
a given sum rate), minimizes the distortions on the receiver side due to reconstruction 
of the source using information from any subsets of the channels. 

1.2.1 Two Descriptions 

The traditional case involves two descriptions as shown in Fig. II. II The total bit rate 
Rt, also known as the sum rate, is split between the two descriptions, i.e. Rt = 
Rq + Ri, and the distortion observed at the receiver depends on which descriptions 
arrive. If both descriptions are received, the resulting distortion (Dc) is smaller than 
if only a single description is received {Dq or Di). It may be noticed from Fig. 11.11 
that Decoder and Decoder 1 are located on the sides of Decoder c and it is therefore 
customary to refer to Decoder and Decoder 1 as the side decoders and Decoder c 
as the central decoder. In a similar manner we often refer to Di,i — 0, 1, as the 
side distortions and Dc as the central distortion. The situation where Dq — Di and 
Rq = Ri is called symmetric MD coding and is a special case of asymmetric MD 
coding, where we allow unequal side rates and unequal side distortions. 

One of the fundamental problems of MD coding is that if two descriptions both 
represent the source well, then, intuitively, they must be very similar to the source and 
therefore also similar to each other Thus, their joint description is not much better 
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Figure 1.1: The traditional two-channel MD system. 



than a single one of them. Informally, we may therefore say that the MD problem 
is about how good one can make the simultaneous representations of the individual 
descriptions as well as their joint description. 

The two-description MD problem was formalized and presented by Wyner, 
Witsenhausen, Wolf and Ziv at an information theory workshop in September 
1979 [50]Q Formally, the traditional two-description MD problem asks what is the 
largest possible set of distortions (Do, -Di, Dc) given the bit rate constraints (i?o, Ri) 
or alternatively the largest set of bit rates {Ro,Ri) given the distortion constraints 
{Dq, Di, Dc)l Both these questions were partially answered by El Gamal and Cover 
who presented an achievable rate-distortion region [42], which Ozarow [107] proved 
was tight in the case of a memoryless Gaussian source and the squared error distortion 
measure. CuiTently, this is the only case where the solution to the MD problem is 
completely known. 



1.2.2 IMany Descriptions 

Recently, the information theoretic aspects of the general case of A' > 2 descriptions 
have received a lot of attention [111, 114, 141, 142, 146]. This case is the natural 
extension of the two-description case. Given the rate tuple (i?o, • ■ • , Rk-i), we seek 
the largest set of simultaneously achievable distortions over all subsets of descriptions. 
The general i^-channel MD problem will be treated in greater detail in Chapter|4] 

With this thesis we show that, at least for the case of audio streaming for lossy 
packet-switched networks, there seems to be a lot to be gained by using more than 
two descriptions. It is likely that this result caiTies over to VoIP and video streaming 
appUcations. 



At that time the problem was already known to several people including Gersho, Ozarow, Jayant, 
Miller, and Boyle who all made contributions towards its solution, see [50] for more information. 
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1.2.3 Scalar vs. Vector Quantization 

In the single-description (SD) case it is known that the scalar rate loss (i.e. the bit 
rate increase due to using a scalar quantizer instead of an optimal infinite-dimensional 
vector quantizer) is approximately 0.2546 bit/dim. [47]. For many applications this 
rate loss is discouraging small and it is tempting to quote Uri Erezo 

"The problem of vector quantization is that scalar quantization works so well." 

However, in the MD case, the sum (or accumulative) rate loss over many descrip- 
tions can be severe. For example, in the two-description case, it is known that the 
scalar rate loss is about twice that of the SD scalar rate loss [136]. Therefore, when 
constructing MD schemes for many descriptions, it is important that the rate loss is 
kept small. To achieve this, we show in this thesis, that one can, for example, use 
lattice vector quantizers combined with an index-assignment algorithm. 



1.3 Contributions 

The MD problem is a joint source-channel coding problem. However, in this work we 
mainly attack the MD problem from a source coding point of view, where we consider 
the general case involving K descriptions. We make extensive use of lattice vector 
quantization (LVQ) theory, which turns out to be instrumental in the sense that the 
proposed MD-LVQ scheme serves as a bridge between theory and practice. In asymp- 
totic cases of high resolution and large lattice vector quantizer dimension, we show 
that the best known information theoretic rate-distortion MD bounds can be achieved, 
whereas in non-asymptotic cases of finite-dimensional lattice vector quantizers (but 
still under high resolution assumptions) we construct practical MD-LVQ schemes, 
which are comparable and often superior to existing state-of-the-art schemes. 

The main contributions of this thesis are the following: 

1 . L-sphere bound for two descriptions 

In the two-channel symmetric case it has previously been established that the 
side descriptions of an MD-LVQ scheme admit side distortions, which (at high 
resolution conditions) are identical to that of L-dimensional quantizers having 
spherical Voronoi cells [120, 139]. In this case we say that the side quantiz- 
ers achieve the i-sphere bound. Such a result has not been established for the 
two-channel asymmetric case before. However, the proposed MD-LVQ scheme 
is able to achieve the L-sphere bound for two descriptions, at high resolution 
conditions, in both the symmetric and asymmetric cases. 



^Said during a break at the International Symposium on Information Theory in Seattle, July 2006. 
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2. MD high-resolution region for three descriptions 

The proposed MD-LVQ scheme appears to be among the first schemes in the 
literature that achieves the largest known high-resolution three-channel MD re- 
gion in the quadratic Gaussian case0 We prove optimality for K < 3 de- 
scriptions, but conjecture optimaUty for any K descriptions. 

3. Exact rate-distortion results for L-dimensional LVQ 

We present closed-form expressions for the rate and distortion performance 
when using L-dimensional lattice vector quantizers. These results are valid 
for smooth stationary sources and squared-error distortion criterion and at high 
resolution conditions. 

4. Rate loss for finite-dimensional LVQ 

The rate loss of the proposed MD-LVQ scheme when using finite-dimensional 
lattice vector quantizers is lattice independent and given by the rate loss of an 
L-sphere and an additional term describing the ratio of two dimensionless ex- 
pansion factors. The overall rate loss is shown to be superior to existing three- 
channel schemes, a result that appears to hold for any number of descriptions. 

5. X-channel asymmetric MD-LVQ 

In the asymmetric two-description case it has previously been shown that by in- 
troducing weights, the distortion profile of the system can range from successive 
refinement to complete symmetric MD coding [27,28]. We show a similar re- 
sult for the general case of K descriptions. Furthermore, for any set of weights, 
we find the optimal number of descriptions and show that the redundancy in 
the scheme is independent of the target rate, source distribution and choice of 
lattices for the side quantizers. Finally, we show how to optimally distribute 
a given bit budget among the descriptions, which is a topic that has not been 
addressed in previous asymmetric designs. 

6. Lattice construction using algebraic ^-modules 

For the two-description case it has previously been shown that algebraic tools 
can be exploited to simplify the construction of MD-LVQ schemes [27, 28, 120, 
139]. We extend these results to A'-channel MD-LVQ and show that algebraic 
^-modules provide simple solutions to the problem of constructing the lattices 
used in MD-LVQ. 



^^A conference version of the proposed symmetric X-channel MD-LVQ scheme appeared in [104] and 
the full version in [105]. The asymmetric /^-channel MD-LVQ scheme appeared in [99]. Independently, 
Chen et al. [16-18] presented a different design of ii'-channel asymmetric MD coding. 
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7. i^-channel MD-LVQ based audio coding 

We present a perceptually robust audio coder based on the modified discrete 
cosine transform and -channel MD-LVQ. This appears to be the first scheme 
to consider more than two descriptions for audio coding. Furthermore, we show 
that using more than two descriptions is advantageous in packet-switched net- 
work environments with excessive packet losses. 

1.4 Structure of Thesis 

The main contributions of this thesis are presented in Chapters |5}{8] and the corre- 
sponding appendices, i.e. AppendiceslEl4Kl 

The general structure of the thesis is as follows: 

Chapter 2 The theory of LVQ is a fundamental part of this thesis and in this chapter 
we describe in detail the construction of lattices and show how they can be used 
as vector quantizers. A majority of the material in this chapter is known, but 
the use of ^-modules for constructing product lattices based on more than two 
sublattices is new. 

Chapter 3 We consider the MD problem from a source-coding perspective and in this 
chapter we cover aspects of SD rate-distortion theory, which are also relevant 
for the MD case. 

Chapter 4 In this chapter we present and discuss the existing MD rate-distortion re- 
sults, which are needed in order to better understand (and to be able to compare 
to) the new MD results to be presented in the forthcoming chapters. 

Chapter 5 Here we present the proposed entropy-constrained X-channel symmetric 
MD-LVQ scheme. We derive closed-form expressions for the rate and distortion 
performance of MD-LVQ at high resolution and find the optimal lattice parame- 
ters, which minimize the expected distortion given the packet-loss probabilities. 
We further show how to construct practical MD-LVQ schemes and evaluate 
their numerical performance. This work was presented in part in [104, 105]. 

Chapter 6 We extend the results of the previous chapter to the asymmetric case. 
In addition we present closed-form expressions for the distortion due to 
reconstructing using arbitrary subsets of descriptions. We also describe how 
to distribute a fixed target bit rate across the descriptions so that the expected 
distortion is minimized. This work was presented in part in [98, 99]. 

Chapter 7 In this chapter we compare the rate-distortion performance of the pro- 
posed MD-LVQ scheme to that of existing state-of-the-art MD schemes as 
well as to known information theoretic high-resolution A'-channel MD rate- 
distortion bounds. This work was presented in part in [98, 102]. 



Section 1.5 List of Papers 



7 



Chapter 8 In this chapter we propose to combine the modified discrete cosine trans- 
form with MD-LVQ in order to construct a perceptually robust audio coder. Part 
of the research presented in this chapter represents joint work with O. Niamut. 
This work was presented in part in [106]. 

Chapter 9 A summary of results and future research directions are given here. 

Appendices The appendices contain supporting material including proofs of lemmas, 
propositions, and theorems. 
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Chapter 

Lattice Theory 



In this chapter we introduce the concept of a lattice and show that it can be used as 
a vector quantizer. We form subsets (called sublattices) of this lattice, and show that 
these sublattices can also be used as quantizers. In fact, in later chapters, we will use 
a lattice as a central quantizer and the sublattices will be used as side quantizers for 
MD-LVQ. We defer the discussion on rate-distortion properties of the lattice vector 
quantizer to Chapters [3] and |4] 

We begin by describing a lattice in simple terms and show how it can be used 
as a vector quantizer. This is done in Section 12.11 and more details can be found in 
Appendixicl Then in Section lZ2l we show that lattice theory is intimately connected to 
algebra and it is therefore possible to use existing algebraic tools to solve lattice related 
problems. For example it is well known that lattices form groups under ordinary vector 
addition and it is therefore possible to link fundamental group theory to lattices. In 
Section |231 we then use these algebraic tools to construct lattices and sublattices. It 
might be a good idea here to consult AppendixlAlfor the definition of Quaternions and 
AppendixiBjfor a brief introduction to module theory. 

We would like to point out that Section ITTI contains most of the essential lattice 
theory needed to understand the concept of MD-LVQ. Sections [2.21 and 1273] are sup- 
plementary to Section 12.11 In these sections we construct lattices and sublattices in 
an algebraic fashion by using the machinery of module theory. This turns out to be a 
very convenient approach, since it allows simple constructions of lattices. This theory 
is therefore also very helpful for the practical implementation of MD-LVQ schemes. 
In addition, we would like to emphasize that by use of module theory we are able 
to prove the existence of lattices which admit the required sublattices and product 
lattices. In Chapters |5H21 we will implicitly assume that all lattices, sublattices, and 
product lattices are constructed as specified in this chapter 
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2.1 Lattices 

An L-dimensional lattice is a discrete set of equidistantly spaced points in the L- 
dimensional Euclidean vector space R^. For example, the set of integers Z forms a 
lattice in M and the Cartesian product Z x Z forms a lattice in R'^. More formally, we 
have the following definition. 

Definition 2.1.1 ([22]). A lattice A C consists of all possible integral linear 
combinations of a set of basis vectors, that is 



where Q E are the basis vectors also known as generator vectors of the lattice. 

The generator vectors — 1, . . . , L, (ot more correctly their transposes) form 
the rows of the generator matrix AI. Usually there exists several generator matrices 
which all lead to the same lattice. In AppendixiDjwe present some possible generator 
matrices for the lattices considered in this thesis. 

Definition 2.1.2. Given a discrete set of points S C M^, the nearest neighbor region 
of s e is called a Voronoi cell, Voronoi region or Dirichlet region, and is defined by 



where ||a;|| denotes the usual norm in M^, i.e. = x'^x. 

As an example. Fig. |2.1(a)| shows a finite region of the lattice A = Z^ consisting 
of all pairs of integers. For this lattice, the Voronoi cells V{X), A £ A, form squares 
in the two-dimensional plane. This lattice is also referred to as the Z'^ lattice or the 
square lattice, cf. Appendix ID. 21 A lattice A and its Voronoi cells y(A),VA G A, 
actually form a vector quantizer When A is used as a vector quantizer, a point x is 
mapped (or quantized) to A e A if a; e V{X). An example of a non-lattice vector 
quantizer is shown in Fig. |2.1(b)| Here we have randomly picked a set of elements 
of M^. Notice that the Voronoi cells are not identical but still their union cover the 
space. On the other hand, in Fig. |2. 1(a)] it may be noticed that the Voronoi cells of A 
are all identical, and we say that each one of them describes a fundamental region. A 
fundamental region of a lattice is a closed region which contains a single lattice point 
and tessellates the underlying space. 

Lemma 2.1.1 ( [22]). All fundamental regions have the same volume i'. 

Lemma 2.1.2 ( [22]). The fundamental volume of A is given by i/ = y/det{A), 
where A = MM'^ is called the Gram matrix. We sometimes write the volume as 
V = det(A). 




(2.1) 



V{s) ^ {x e : - sf <\\x- s'f, Vs' G S}, 



(2.2) 
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Let us define Vq = V{0), i.e. the Voronoi cell around the lattice point located at 
the origin. This region is called a fundamental region of the lattice since it specifies 
the complete lattice through translations. We then have the following definition. 

Definition 2.1.3 ( [22]). The dimensionless normalized second moment of inertia 
G(A) of a lattice A is defined by 

G(A) ^ jj^ J M'd., (2.3) 

J Vq 

where i' is the volume of Vq. 

Applying any scaling or orthogonal transform, e.g. rotation or reflection on A will 
not change G{A), which makes it a good figure of merit when comparing different 
lattices (quantizers). Furthermore, G(A) depends only upon the shape of Vo, and in 
general, the more sphere-like shape, the smaller normalized second moment [22]. 

2.1.1 Sublattices 

If A is a lattice then a sublattice A' C A is a subset of the elements of A that is itself 
a lattice. For example if A = Z then the set of all even integers is a sublattice of A. 
Geometrically speaking, a sublattice A' C A is obtained by scaling and rotating (and 
possibly reflecting) the lattice A so that all points of A' coincide with points of A. A 
sublattice A' C A obtained in this manner is referred to as a geometrically-similar 
sublattice of A. Fig. 12. 21 shows an example of a lattice A C and a geometrically- 
similar sublattice A' C A. In this case A is the hexagonal lattice which is described in 
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Appendix ID. 3 1 It may be noticed from Fig. 12. 21 that all Voronoi cells of the sublattice 
A' contain exactly seven points of A. In general we would like to design a sublattice so 
that each of its Voronoi cells contains exactly N points of A. We call N the index value 
of the sublattice and usually write it as = |A/A'|. Normalizing A^ by dimension, 
i.e. N' = N^/^, gives what is known as the nesting ratio. We call a sublattice A' C A 




-3-2-1 1 2 3 



Figure 2.2: The hexagonal lattice A (small dots) and a sublattice A' C A (circles) of index 
N = 7. The solid lines illustrate the boundaries of the Voronoi cells ofA'. 

clean if no points of A lies on the boundaries of the Voronoi cells of A'. For example, 
the sublattice of Fig. 12.21 is clean. If A' C A is a clean sublattice we call the index 
= I A/ A' I an admissible index value. In this work we are mainly interested in 
clean sublattices and we will further discuss the issue of admissible index values in 
Section lZTT] 

2.2 ^-Lattice 

We showed in the previous section that, geometrically speaking, an L-dimensional 
lattice A C is a discrete set of regularly spaced points in M^. From Appendix iBl 
it can be deduced that, algebraically speaking, an L-dimensional lattice A C is 
a free (torsion-free) discrete ^-module of rank L with compact quotient M^/A. In 
this section we will consider the latter definition of a lattice and construct lattices and 
sublattices by use of the theory of modules. 

A lattice A C as defined in ( 12.11 ) forms an additive group (A, +) under or- 
dinary vector addition with the zero-vector being the identity element. If the group 
further admits left or right multiplication by the ring ^ then we call A a ^-module. 
In other words, A is a ^-module if it is closed under addition and subtraction of 
members of the group and closed under scalar multiplication by members of the ring. 
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see Appendix iB] for details. Since A is also a lattice we sometimes prefer the name 
^-lattice over ^-module. 

Let = 1 , . . . , L be a set of linearly independent vectors in and let ^ C M 
be a ring. Then a left ^-lattice A generated hy Q,i — 1, . . . ,L consists of all linear 
combinations 

CiCi + '-' + aa, (2.4) 

where ^ ,i — 1, . . . , L [22]. A right ^-lattice is defined similarly with the 
multiplication of Q on the right by £,i instead. 

We have so far assumed that the underlying field is the Cartesian product of the 
reals, i.e. M^. However, there are other fields which when combined with well defined 
rings of integers will lead to ^-lattices that are good for quantization. Let the field be 
the complex field C and let the ring of integers be the Gaussian integers where [22] 

^ = {a + «6 : a, 6 e Z}, z = V^. (2.5) 

Then we may form a one-dimensional complex lattice (to which there always exists 
an isomorphism that will bring it to M^) by choosing any non-zero element (a basis) 
Ci € C and insert in ( I2.4l i. cf. Fig. [23(a)] where we have made the arbitrary choice of 
Ci — 11.2 — 2.3i. The lattice described by the set of Gaussian integers is isomorphic 
to the square lattice — 1? . The operation ^^^i then simply rotate and scale the 
lattice. To better illustrate the shape of the ^-lattice we have in Fig. |2.3(a)| also 
shown the boundaries (solid lines) of the nearest neighbor regions (also called Voronoi 
cells) between the lattice points. Fig. |2.3(b)| shows an example where the basis Qi = 
11.2 — 2. 3i has been multiplied by the Eisenstein integers where [22] 

^ = {6 + : 6 , 6 e , ^ = e'"^/' . (2.6) 
The ring of algebraic integers ^ is defined by [22] 

^ = Ui+wi6 :ei,6ez}, (2.7) 

where uji is, for example, one of 



(2.8) 



Figs . |2 . 3 ("c)| and [Z3 (d)| show examples where ^ is the ring of algebraic integers and 
where cji = \/— 5 and oji — (— 1 + \/~7)/2, respectively. In both cases we have used 
the basis Ci = 11.2 - 2.3i. 

2.2.1 ^-Sublattice 

If A' is a submodule of a ^-module A then A' is simply a sublattice of the lattice A. 
More formally we have the following lemma. 
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Figure 2.3: One-dimensional complex ^ -lattices constructed from different rings of integers 
by use of the basis C,i = 11.2 — 2.3j. The solid lines illustrate the boundaries of the nearest 
neighbor regions (Voronoi cells) between lattice points. 



Lemma 2.2.1 ( [1]). Let ^ be a ring. If A is ^-module and A' C A, A' 7^ 0, then 
A' is a ^-submodule of A if and only if ^lA'^ + ^2^2 G A' for all A'^ , G A' and 

Let A be a ^-module. Then we may form the left submodule A' = ^A and 
the right submodule A" = A^ by left (or right) multiplication of A by ^ e ^ . 
For example let A be the ^-module given by the Eisenstein integers, i.e. K — 
This lattice can be regarded as a two-dimensional real lattice in in which case 
it is usually referred to as A2. Then let us form the submodule A' = ^A where 
^ — —3 — 2uj and lo — e^^''^^, see Fig. 12.41 When the modules in question are lattices 
we will usually call A a ^-lattice and A' a ^-sublattice. Sometimes when the ring 
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^ is clear from the context or irrelevant we will use the simpler terms lattice and 
sublattice for A and A', respectively. 
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Figure 2.4: The Eisenstein lattice A (\s here shown with dots and the circles illustrate points of 
the sublattice A' = ^A, ^ = —3 — 2lu, lo = e^'^^^^ . The solid lines describe the boundaries 
between neighboring Voronoi cells of the sublattice points. Notice that there are 7 dots in each 
Voronoi cell. The points marked with squares are the seven coset representatives of the quotient 
A/A'. 



2.2.2 Quotient Modules 

In this section we consider quotient modules and the next section is concerned with 
group actions on these quotient modules. Although perhaps unclear at this point, we 
show later that these concepts are important in order to identify or associate a set of 
sublattice points with a given lattice point. This identification process, which we call 
either the labeling problem or the problem of constructing an index assignment map, 
focuses on labeling the coset representatives of A/ A', i.e. the quotient module. It 
then turns out, as first observed in [120, 139], that we actually only need to label the 
representatives of the orbits of A/A'/Tm instead of all coset representatives of A/A'. 
Further details about quotient modules and group actions are given in AppendixiBl 

Definition 2.2.1. Let A be a ^-module and A' a ^-submodule of A. Then A' 
induces a partition A/ A' of A into equivalence classes (or cosets) modulo A'. We call 
such a partition the quotient module. 
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The order or index |A/A'| of the quotient module A/ A' is finite and each element 
of A/ A' is a representative for an infinite set called a coset. For any A e A the coset 
of A' in A determined by A is the set A + A' = {A + A' : A' £ A'}. In this work 
we always let the group operation be ordinary vector addition which is a commutative 
operation so that the left and right cosets coincide. As such there is no ambiguity with 
respect to left and right cosets when referring to the coset A + A'. We will use the 
notation [A] when referring to the coset A + A' and we call A the coset representative. 
It should be clear that any member of the coset [A] can be the coset representative. 
To be consistent we will always let the coset representative be the uniqu^ vector of 
[A] which is in the Voronoi cell of the zero-vector of A'. For example if A and A' are 
defined as in Fig. 12. 41 then the index |A/A'| = 7 and there is therefore seven distinct 
cosets in the quotient module A/ A'. The seven cosets representatives are indicated 
with squares in Fig. 12.41 

2.2.3 Group Actions on Quotient Modules 

Let Tm C Aut(A) be a group of order m of automorphisms of A. We then denote the 
set of orbits under the action of Tm on the quotient module A/ A' by A/A'/r,„ . For 
example let r2 = {I2, —^2} be a group (closed under matrix multiplication) of order 
2, where I2 is the two-dimensional identity matrix. Let the ^-module A be identical 
to and let A' be a submodule of A of index N = 81. In other words, there are N 
coset representatives in the quotient module A/ A' whereas the set of orbits K/ K' /V2 
has cardinality |A/A'/r2| = 4L This is illustrated in Fig. |2.5(a)| where the coset 
representatives of A/ A' are illustrated with dots and representatives of the orbits of 
A/A'/r2 are marked with circles. 
Next consider the group given by 

r.. -;)}, 

which has order 4 and includes r2 as a subgroup. Fig. |2.5(b)| shows coset repre- 
sentatives for A/A' and representatives for the set of orbits A/A'/r4. Notice that 
|A/A'/r4|=2L 

2.3 Construction of Lattices 

We now show how to construct the lattices and sublattices that later will be used as 
quantizers in MD-LVQ. 



'^We will later require that A' is a clean sublattice of A from which the uniqueness property is evident. 
If A' is not clean then we make an arbitrary choice amongst the candidate representatives. 
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Figure 2.5: The 81 coset representatives for A/ A' are here shown as dots and representatives 
for the orbits of (a) A/A'/Fa and (b) A/A'/r4 are shown as circles. 



2.3.1 Admissible Index Values 

For any geometrically-similar sublattice A' of A, a number of lattice points of A will 
be located within each Voronoi cell of A' and perhaps on the boundaries between 
neighboring Voronoi cells. In the latter case ties must be broken in order to have well 
defined Voronoi cells. To avoid tie breaking it is required that A' has no lattice points 
on the boundary of its Voronoi cells. In this case. A' is said to be clean. As previously 
mentioned, we call an index value of a clean sublattice an admissible index value. 
In [21] partial answers are given to when A contains a sublattice A' of index N that 
is geometrically-similar to A, and necessary and sufficient conditions are given for 
any lattice in two dimensions to contain a geometrically-similar and clean sublattice 
of index N. These results are extended in [28] to geometrically-similar and clean 
sublattices in four dimensions for the and D4 lattice. In addition, results are given 
for any lattice where L = Ak, k > 1. Table [TTI briefly summarizes admissible 
index values for the known cases. In general has a geometrically-similar and 
clean sublattice if and only if N is odd and 

a) L odd and N an L*'* power, or 

h) L = 2 and N of the form 0^ + 6^, or 

c) L = 4fc, fc > 1 and N of the form m^^^ for some integer m, 
see [28] for details. 

It can be shown that squaring an admissible index value yields another admissible 
index value for all lattices considered in this work. We can generalize this even fur- 
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Lattice 



Dim. 



Admissible index values 



Z 



1 
2 
2 
4 
4 



1,3,5,7,9,... 

1,5,9,13,17,25,29,37,41,45,49,. . . 
1,7,13,19,31,37,43,49,... 
1,25,49,169,289,625,... 
1,25,49,81,121,169,225,289,361,. . . 



Table 2.1: Admissible index values for geometrically-similar and clean sublattices in one, two 
and four dimensions. See Appendix\D\for more information about these sets of index values. 

ther and show that the product of any number of admissible index values leads to an 
admissible index value. 

Lemma 2.3.1. For the lattices Ai,D^ and where L = 1,2 or L = 4fc, where 
A; > 1, the product of two or more admissible index values yields another admissible 
index value. 

Proof. See Appendix|E] □ 

As noted in [21] it is always possible by e.g. exhaustive search to see if a lattice 
A contains a sublattice A' with an index value of iV = c^/^, c G M+. Let the Gram 
matrix of A be A. Then search through A to see if it contains a set of generator 
vectors with Gram matrix cA. In large lattice dimensions this approach easily becomes 
infeasible. However, for known lattices the admissible index values can be found off- 
line and then tabulated for later use. 

If two lattices A C and A' C are concatenated (i.e. their Cartesian prod- 
uct is formed) then the resulting lattice A" is of dimension L" = L + L', cf. Defini- 
tion lC.l.Tl The set of admissible index values of A" (when normalized by dimension) 
might be different than that of A or A'. For example let A = where the admissible 
index values are the odd integers. Then notice that the four-dimensional Z"* lattice is 
simply a cascade of four Z^ lattices. However, the admissible index values (normal- 
ized per dimension) of Z'^ are given by (see Appendix lD.4l l 

N' = {1, 2.24, 2.65, 3, 3.32, 3.61, 3.87, 4.12, 4.36, 4.58, 4.8, 5, . . . }, (2.10) 

where we have shown the index values of in boldface. Thus, by forming a higher 
dimensional lattice by cascading smaller dimensional lattices it is possible to achieve 
more (or at least different) index values. 

A different strategy is to change the underlying ring ^ as shown in Fig. l2.3l which 
results in a different lattice of the same dimension that might lead to new index val- 
ues. In this thesis, however, we will be using the known admissible index values of 
TableO 
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2.3.2 Sublattices 

In this section we construct sublattices and primarily focus on a special type of sub- 
lattices called product lattices. In [28] the following definition of a product lattice was 
presented. 

Definition 2.3.1 ( [28]). Let ^ be an arbitrary ring, let A = ^ and form the two 
sublattices Ao = ^oA and Ai = A^i, e A, i = 0, 1. Then the lattice A^r = CoA^i 
is called a product lattice and it satisfies Ajr C A;, i = 0, 1. 

In this work, however, we will make use of a more general notion of a product lattice 
which includes Definition 12. 3. II as a special case. 

Definition 2.3.2. A product lattice A^r is any sublattice satisfying A^r C A^ where 
A, = ^,A or A, = A^,, i = 0, . . . , if - 1. 

The construction of product lattices based on two sublattices as described in 
Definition 12 . 3 . 1 1 was treated in detail in [28]. In this section we extend the existing 
results of [28] and construct product lattices based on more than two sublattices for 
L = 1,2 and 4 dimensions for the root lattices Z^, Z^, v42, Z'' and 1)4, which are 
described in Appendix [D] Along the same lines as in [28] we construct sublattices 
and product lattices by use of the ordinary rational integers Z as well as the Gaussian 
integers , Eisenstein integers S, Lipschitz integral Quaternions Mt^ and the Hurwitz 
integral Quaternions where ^ and S are given by ( 12.51 ) and ( 12. 6t , respectively, 
and [22] 

^0 = {ii + iii + Jii + : 6, 6, 6, ^4 e Z}, (2.1 1) 

= {Ci + ^6 + j6 + Hi ■■ 6,6,6, 6 all in Z or all in Z + 1/2}, (2.12) 

where i, j and fc are unit Quaternions, see Appendix lAl for more information. For 
example a sublattice Ai of A = Z is easily constructed, simply by multiplying all 
points A € A by ^ where ^ e Z\{0}|f| This gives a geometrically-similar sublattice 
Ai = ^Z of index |^|. This way of constructing sublattices may be generalized by 
considering different rings of integers. For example, for the square lattice K — 
whose points lie in the complex plane, a geometrically-similar sublattice of index 2 
may be obtained by multiplying all elements of A by the Gaussian integer ^ = 1 + i. 

Sublattices and product lattices of Z^, and A2 

The construction of product lattices based on the sublattices Z^^Z^ and Ai is a 
straight forward generalization of the approach taken in [28]. Let the lattice A be 
any one of Z^ — Z"^ — or A2 — S and let the geometrically-similar sublattices 
Ai be given by 6A where 6 is an element of the rational integers Z, the Gaussian 
integers ^ or the Eisenstein integers S, respectively. 

^Since A is a torsion free ^-module the submodule A' = gA is a non-trivial cyclic submodule when- 
ever ^ ^ e A. 
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Lemma 2.3.2. A^r = ^oCi ■ ' • fx-i A is a product lattice. 

Proof. See Appendix|E] □ 

Also, as remarked in [28], since the three rings considered are unique factorization 
rings, the notion of least common multiple (1cm) is well defined. Let us define — 
lcm(^o, . . . , i^_R--i) so that £,i\£,n, i.e. £,i divides ^n- This leads to the following lemma. 

Lemma 2.3.3. A^ = A is a product lattice. 

Proof. See Appendix|E] □ 

The relations between A, A^, A^ and A^ as addressed by Lemmas |2.3.2| and |2.3.3l 
are shown in Fig.|22] For example, let A = $^(= Z^) and let A^o = 45 and A^i = 81. 
Then we have that lcm(45, 81) = 405 and 45 • 81 = 3645. We may choose = 
3 + 6i,Ci = 9 and = 9 + 18i, so that |^oP 45, = 81 and \^n? = 405. 
Notice that ^o|?n and ^i|^n, i.e. |^ = 3 G ^ and |^ = 1 + 2i e ^. Since both 
and ^1 divides the lattice An — will be a sublattice of Ao ~ £,oA as well as 
Ai = ^lA, see Fig. O 

A 




= ^oCi---^x-iA 

Figure 2.6: The intersection (meet) of K arbitrary sublattices form a product lattice for , 
and A2. 



Sublattices and product lattices of 

As was done in [28] we will use the Quaternions [71, 150] for the construction of 
sublattices and product lattices for Z^. The Quaternions form a non-commutative ring 
and it is therefore necessary to distinguish between left and right multiplication [71, 
150]. For the case of two sublattices we adopt the approach of [28] and construct the 
sublattice Ao by multiplying A on the left, i.e. Aq = and Ai is obtained by right 
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Figure 2.7: The lattice K = is here shown as dots. The two lattices Ao = (3 + 6j)A 
(squares) and Ai — 9A (circles) are sublattices of K and the lattice An = (9 + 18j)A (stars) 
is a sublattice of all the lattices Aq , Ai and A. The solid lines describe the boundary of the 
Voronoi cell Vo of the product lattice point located at the origin. 



multiplication Ai = A^i, see Fig. 12. 81 More than two descriptions was not considered 
in [28]. Let the K sublattices be of index iVo, . . . , Nk-i respectively. Then we may 
form Ao = ^oA and Ai = A^i as above. However, by letting A2 — A^2 we run into 
trouble when creating the product lattice. For example, if we define A^r = CoA^i^2 it 
is clear that A^r C Aq and A^r C A2. The problem is that in general K.^ ^ Ai since 
7^ ^2^1 and we therefore have to restrict the set of admissible index values. 

Lemma 2.3.4. Let iVo and iVi be admissible index values for Z'^. Then Nq and 
(which are admissible index values for Z*), can be associated with a pair of Lipschitz 
integers (Co, ^i) that commute, i.e. ^oCi = Ci^o- 

Proof. See Appendix|E] □ 

From Lemma 12.3.41 it follows that there exist an infinite number of pairs of ad- 
missible index values [N^^Ni) where iVo 7^ Ni such that the Lipschitz integers 
Co and Ci commute. For example, let Nq — 7^, A^i — 13^, Af2 = 5^ and define 

Ao CoA, Ai = ACi and A2 = AC2 whereto = -2-«-j-fc,Ci = -2-i + Oj + OA: 
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and ^2 = — 3 — 2i + Qj + Ok. For this example we have ^o^i 7^ Ci^o ^nd ^0^2 7^ ^2^0 
but ^1^2 — £,2^1- Letting A^r = ^oA^i^2 niakes sure that A^ C A.; for i = 0,1, 2, 
since A^^ ~ (CoA^i)52 = (CoA^2)Ci- I" general it is possible to construct the prod- 
uct lattice Att such that A^r C A^ for i — 0, . . . , if — 1 as long as any i^T — 1 of 
the K ^i's commute, see Fig. 12.91 where — lcm(^i, . . . , £,k-i)- If all the pairs 
(Cii CjOi hj G {Oi ■ ■ ■ ,K ~ 1} commute the procedure shown in Fig. l2.6l is also valid. 



A 




A^ = ^oACi---Ck-i 



Figure 2.9: The intersection (meet) of K arbitrary sublattices form a product lattice for . 
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Sublattices and product lattices of 

For D4 we use the ring of Hurwitzian integers, i.e. S J^i. For the case of two sub- 
lattices we design the sublattices and product lattices as in [28] and shown in Fig. 12.81 
For more than two sublattices we have to make a restriction on the set of allowable 
admissible index values. The Quaternions leading to admissible index values for D4 
obtained in [28] are of the forrrOCi = f (1 + + |(j + ^) € -^i, where a and b are 
odd positive integers. Quaternions of this form do generally not commute since both 
a and b are nonzero. In fact two Quaternions commute if and only if their vector parts 
are proportional [6], i.e. linearly-dependent, which rarely happens for the Quaternions 
of the form — f (1 + «) + | (j + fc) g J^i . For example we did an exhaustive search 
based on all admissible index values between 25 and 177241 and found only five pairs 
(up to permutations) of Quaternions that commute. These are shown in Table 12.21 



25 


15625 


1 
2 


f iiH 




|fc 


2 ^ 




f^- + 


15 1, 
2 


169 


105625 


1 
2 


f |zH 






2 ^ 




fj- + 


25 I, 

~2~ A, 


625 


28561 


5 
2 


f |zH 




2^ 


2 ^ 


h 4 




13 1. 
- -j-K 


625 


83521 


5 
2 


f |i H 






2 ^ 






- -2-fc 


28561 


83521 


2 ' 






13 jL 

- —ft 


2 ^ 


hfz4 


-¥^•4 


- —k 



Table 2.2: £ac/! raw shows two Quaternions airf which commute, i.e. i^o^i = CiCo- 

We therefore restrict the set of admissible index values to Ni E {a, b} for i ~ 
0, . . . , K —1 where a and b are any two admissible index values. With this the product 
lattice, for K > 2 sublattices, is based on only two integers e.g. ^0 and ^1 as shown in 
Fig. |2.8| and the index of the product lattice is then N^^ — ab. With this approach it is 
possible to obtain Ajr C for i — 0, . . . , K ~ 1. 



''With two exceptions being = | + |i + |j + ffc and 6 = | + f « + |i + ffc both leading to 
an index value of A'^ = 49. 



Chapter 



Single-Description Rate-Distortion 
Theory 

Source coding with a fidelity criterion, also called rate-distortion theory (or lossy 
source coding), was introduced by Shannon in his two landmark papers from 
1948 [121] and 1959 [122] and has ever since received a lot of attention. For an 
introduction to rate-distortion theory we refer the reader to the survey papers by Ki- 
effer [74] and Berger and Gibson [9] and the text books by Berger [8], Ciszar and 
Korner [26] and Cover and Thomas [24]. 

3.1 Rate-Distortion Function 

A fundamental problem of rate-distortion theory is that of describing the rate R 
required to encode a source X at a prescribed distortion (fidelity) level D. Let 
= {Xi}, i ^ 1, .... L be a sequence of random variables (or letters) of a station- 
ar}0random process X. Let X be the reproduction of X and let x and x be realizations 
of X and X, respectively. The alphabets ^ and ^ of X and X, respectively, can be 
continuous or discrete and in the latter case we distinguish between discrete alphabets 
of finite or countably infinite cardinality. When it is clear from context we will often 
ignore the superscript ^ which indicates the dimension of the variable or alphabet so 
that X e ^ C denotes an L-dimensional vector or element of the alphabet ^ 
which is a subset of M^. 

Definition 3.1.1. A fidelity criterion for the source X is a family p'^^\X,X),L £ 
N of distortion measures of which p'^^) computes the distortion when representing 

'Throughout this work we will assume all stochastic processes to be discrete-time zero-mean weak- 
sense stationary processes (unless otherwise stated). 
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X by X. If = j:T,LlP{X^,Xi) then p is said to be a single-letter 

fidelity criterion and we will then use the notation p{X, X). Distortion measures of 
the form p{X — X) are called difference distortion measures. For example p{X, X) — 
— Xjpisa difference distortion measure (usually referred to as the squared-error 
distortion measure). 

In this work we will be mainly interested in the squared-error single-letter fidelity 
criterion which is defined by 



1 ^ 

p{X,X)^-Y,{X^-X,)\ (3.1) 

i=l 

With this, formally stated. Shannon's rate-distortion function R{D) (expressed in 
bit/dim.) for stationary sources with memory and single-letter fidelity criterion, p, is 
defined as [8] 

R{D) = lim RLiD), (3.2) 

L — >oo 

where the L*'* order rate-distortion function is given by 

Rl{D)=M{^I{X;X) : Ep{X,X) < D}, (3.3) 

where I{X;X) denotes the mutual informatioij^ between X and X, E denotes the 
statistical expectation operator and the infimum is over all conditional distributions 
fx\x^^\^) for which the joint distributions ^(x, i:) = f ^[x) f j.^j^{x\x) satisfy 
the expected distortion constraint given by 

The L*'* order rate-distortion function Rl{D) can be seen as the rate-distortion func- 
tion of an L-dimensional i.i.d. vector source X producing vectors with the distribution 
ofX [82]. 

Let h{X) denote the differential entropy (or continuous entropy) of X which is 
given by [24] 

h{X) = - f f^{x)\og,{f^{x))dx 

J X 

and let the differential entropy rate h{X) be defined by h{X) = limi^oo j^h{X) 
where for independently and identically distributed (i.i.d.) scalar processes h{X) = 



marginals and respectively, is defined as [24] 

^ x,x (■^' 



I{X;X)= I / . -^(x,£)log2 ( , 1 dxdx. 
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j^h{X). With a slight abuse of notation we will also use the notation h{X) to indicate 
the dimension normalized differential entropy of an i.i.d. vector source. If p is a dif- 
ference distortion measure, then ( I3.2l i and (I3.3l l can be lower bounded by the Shannon 
lower bound [8] . Specifically, if E pis the mean squared error (MSE) fidelity criterion, 
then [8, 82] 

R{D)>h{X)-^\og^{27reD), (3.5) 

where equality holds at almost all distortion levels D for a (stationary) Gaussian 
source [8]0 In addition it has been shown that (13.5b becomes asymptotically tight 
at high resolution, i.e. as £> ^ 0, for sources with finite differential entropies and 
finite second moments for general difference distortion measures, cf. [82]. 

Recall that the differential entropy of a jointly Gaussian vector is given by [24] 

MX) = ilog2((27re)^|<l>|), (3.6) 

where \^\ is the determinant of $ = EXX'^, i.e. the covariance matrix of X. It 
follows from ( 13.51 ) that the rate-distortion function of a memoryless scalar Gaussian 
process of variance cr^ is given by 

RiD) = ^log, (^^y (3.7) 

whenever D < a\ and R{D) — for D > a\ since R{D) is everywhere non- 
negative. 

The inverse of R{D) is called the distortion-rate function D{R) and it basically 
says that if a source sequence is encoded at a rate R the distortion is at least D{R). 
From (I3.7l i we see that the distortion-rate function of the memoryless Gaussian process 
is given by 

D{R) = ai2-2«, (3.8) 
which is shown in Fig. |3.1| for the case of a\ = 1. 

Remark 3.1.1. From ( 13. 8t and also from Fig. 13. H it may be seen that each extra bit 
reduces the distortion by a factor of four — a phenomena often referred to as the "6 
dB per bit rule" [60]. In fact, the "6 dB per bit rule" is approximately true not just for 
the Gaussian source but for arbitrary sources. 

The rate-distortion function of a memoryless scalar source and squared-error dis- 
tortion measure may be upper and lower bounded by use of the entropy-power in- 
equality, that is [8] 

ilog.(f)>i?(i?)>ilog,(§), (3.9) 

'Eq. ii.5\ is tight for all -D < ess inf Sx , where Sx is the power spectrum of a stationary Gaussian 
process X [8]. 
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R [bit/dim.] 

Figure 3.1: D[R) for the unit-variance memoryless Gaussian source. 

where Px is the entropy powerM Similarly, the distortion-rate function is bounded as 

Px2-^" < D{R) < cr|2^2^, (3.10) 

with equalities all the way in both ( 13.9b and ( 13.10b if X is Gaussian. 

Remark 3.1.2. Inequalities ( 13.9b and (13.10b show that, of all sources, the Gaussian 
source is the hardest to compress. 

3.2 Quantization Theory 

A quantizer Q consists of a set of decision cells ,5^ = {Si : i € J^} where C N 
together with a set of reproduction values = {c^ : i e J^} [60]. The operation of 
quantization is defined as Q{x) = Ci if x e Si. We require that ^ cover the input 
space ^ which implies that IJ^^ ^ Si D ^ and often we need to partition ^ so 
that sets of y are pairwise disjoint, i.e. Si D Sj — 9,i ^ j so that IJ^^ ^ Si = . 

Definition 3.2.1. The decision cells of a nearest neighbor quantizer are called Voronoi 
cells, Voronoi regions or Dirichlet regions [22]. Given the i*'' reproduction value Ci 
the Voronoi cell V{ci) is defined by 

V{cC) ^{xe.r : p(x,c,) < pix,c,), Vj e y}, (3.11) 

'"The entropy power Px = (27re)-l22^(-^) of a source X is defined as the variance of a Gaussian 
density that has the same differential entropy as X [8]. 
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where ties (if any) can be arbitrarily brokenly 

It follows that the expected distortion of a quantizer is given by 

-Dq = ^ / fx{x)p{x,Ci)dx. (3.12) 

Let us for the moment assume that ^ = IR-^ and 'if — ^ C M.^. Then, for 
the squared error distortion measure, the Voronoi cells of an L-dimensional nearest 
neighbor quantizer (vector quantizer) are defined as 

V{x,)^{xeR^ :\\x-x,\\^ <\\x-Xjf,yxj e .^}, € JT, (3.13) 

where || • || denotes the f2-norm, i.e. = J2n=i ^n- 

Vector quantizers are often classified as either entropy-constrained quantizers or 
resolution-constrained quantizers or as a mixed class where for example the output 
of a resolution-constrained quantizer is further entropy coded0 When designing an 
entropy-constrained quantizer one seeks to form the Voronoi regions V{xi),Xi G 3(J , 
and the reproduction alphabet ^ such that the distortion Dq is minimized subject to 
an entropy constraint R on the discrete entropy H{X). Recall that the discrete entropy 
of a random variable is given by [24] 

HiX) - - ^ P(iO log2(P(x,)), (3.14) 

where P denotes probability and P{xi) = P{x G V{xi)). On the other hand, in 
resolution-constrained quantization the distortion is minimized subject to a constraint 
on the cardinality of the reproduction alphabet. In this case the elements of are 
coded with a fixed rate of i? logjd^D/i. For large vector dimensions, i.e. when 
L ^ 1, it is very likely that randomly chosen source vectors belong to a typical set 
^(^) in which the elements are approximately uniformly distributed [24]. As a con- 
sequence, in this situation there is not much difference between entropy-constrained 
and resolution-constrained quantization. 

There exists several iterative algorithms for designing vector quantizers. One 
of the earliest such algorithms is the Lloyd algorithm which is used to construct 
resolution-constrained scalar quantizers [87], see also [88]. The Lloyd algorithm is 
basically a cyclic minimizer that alternates between two phases: 

1. Given a codebook "^^ ^ find the optimal partition of the input space, i.e. 
form the Voronoi cells V{xi)yxi e 

"Two neighboring L-dimensional Voronoi cells for continuous-alphabet sources share a common L'- 
dimensional face where L' < L — 1. For discrete-alphabet sources it is also possible that a point is equally 
spaced between two or more centroids of the codebook, in which case tie breaking is necessary in order to 
make sure that the point is not assigned to more than one Voronoi cell. 

'^Entropy-constrained quantizers (resp. resolution-constrained quantizers) are also called variable-rate 
quantizers (resp. fixed-rate quantizers). 
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2. Given the partition, form an optimal codebook, i.e. let Xi G ^ he the centroid 

of the set x G V{xi). 

If an analytical description of the pdf is unavailable it is possible to estimate the pdf 
by use of empirical observations [45]. Furthermore, Lloyd's algorithm has been ex- 
tended to the vector case [45, 81] but has not been explicitly extended to the case of 
entropy-constrained vector quantization. Towards that end Chou et al. [19] presented 
an iterative algorithm based on a Lagrangian formulation of the optimization problem. 
In general these empirically designed quantizers are only locally optimal and unless 
some structure is enforced on the codebooks, the search complexity easily becomes 
overwhelming (the computational complexity of an unconstrained quantizer increases 
exponentially with dimension) [45]. There exists a great deal of different design al- 
gorithms and we refer the reader to the text books [22,45,57] as well as the in-depth 
article by Gray and Neuhoff [60] for more information about the theory and practice 
of vector quantization. 



3.3 Lattice Vector Quantization 

In this work we will focus on structured vector quantization and more specifically on 
lattice vector quantization (LVQ) [22,46, 57] . A family of highly structured quantizers 
is the tesselating quantizers which includes lattice vector quantizers as a sub family. 
In a tesselating quantizer all decision cells are translated and possibly rotated and 
reflected versions of a prototype cell, say Vq. In a lattice vector quantizer all Voronoi 
cells are translations of Vq which is then taken to be Vq = T^(0), i.e. the Voronoi cell of 
the reproduction point located at the origin (the zero vector) so that V{xi) = Vb + 
In a high-resolution lattice vector quantizer the reproduction alphabet ^ is usually 
given by an L-dimensional lattice A C M^, see Appendices ICl and ID] for more details 
about lattices. 

In order to describe the performance of a lattice vector quantizer it is convenient to 
make use of high resolution (or high rate) assumptions which for a stationary source 
can be summarized as follows [45,57]: 

1 . The rate or entropy of the codebook is large, which means that the variance of 
the quantization error is small compared to the variance of the source. Thus, the 
pdf of the source can be considered constant within a Voronoi cell, i.e. (x) « 
f^{xi) if X e V{xi). Hence, the geometric centroids of the Voronoi cells are 
approximately the midpoints of the cells 

2. The quantization noise process tends to be uncorrelated even when the source 
is correlated 

'^Notice that not all tesselating quantizers are lattice quantizers. For example, a tesselating quantizer 
having triangular shaped decision cells is not a lattice vector quantizer 
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3. The quantization error is approximately uncorrelated with the source 

Notice that 1) is always true if the source distribution is uniform. Furthermore, all 
the above assumptions have been justified rigorously in the limit as the variance of 
the quantization error tends to zero for the case of smooth sources (i.e. continuous- 
alphabet sources having finite differential entropies) [86, 145, 161]. If subtractive 
dither is used, as is the case of entropy-constrained dithered (lattice) quantization 
(ECDQ), the above assumptions are valid at any resolution and furthermore the quan- 
tization errors are independent of the source [159, 160, 165]. A nice property of ECDQ 
is that the additive noise model is accurate at any resolution, so that the quantization 
operation can be modeled as an additive noise process [160]. The dither signal of an 
ECDQ is an i.i.d. randonQ process which is uniformly distributed over a Voronoi cell 
of the quantizer so that for a scalar quantizer the distribution of the quantization errors 
is uniform. Asymptotically, as the dimension of the ECDQ grows unboundedly, any 
finite-dimensional marginal of the noise process becomes jointly Gaussian distributed 
and the noise process becomes Gaussian distributed in the divergenc sense [161]. 
These properties of the noise process are also valid for entropy-constrained LVQ 
(without dither) under high resolution assumptions [161]. It is interesting to see that at 
very low resolution, i.e. as the variance of the quantization error tends to the variance 
of the source, the performance of an entropy-constrained scalar quantizer is asymptot- 
ically as good as any vector quantizer [91]. 

3.3.1 LVQ Rate-Distortion Theory 

Let H denote the discrete entropy H{^) of the codebook of an entropy-constrained 
vector quantizer and let the dimension-normalized MSE distortion measure -Dl be 
defined as 

DL^^E\\X-Xf. (3.15) 

Then by extending previous results of Bennett [7] for high resolution scalar quantiza- 
tion to the vector case it was shown by Zador [156] that if X has a probability density 
ther0 

lim i?i22^/^ = aL22''W/^, (3.16) 

where h{X) is the differential entropy of X and is a constant that depends only on 
L. In the scalar case where L = 1 it was shown by Gish and Pierce [47] that ai — 1/12 

'^The dither signal is assumed known at the decoder so it is in fact a pseudo-random process. 
'^The information divergence (also called Kullback-Leibler distance or relative entropy) between two 
pdfs fx and gx is defined as [24] 

mWa) = fx{x)\og^(fx(x)/gx{^))dx. 

"Later on the precise requirements on the source for j3.16t to be valid was formalized by Linder and 
Zeger [86]. 
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and that the quantizer that achieves this value is the unbounded uniform scalar (lattice) 
quantizer. For the case of 1 < L < oo the value of is unknown [86] It was 
conjectured by Gersho in 1979 [44] that if the source distribution is uniform over 
a bounded convex set in then the optimal quantizer will have a partition whose 
regions are all congruent to some polytope. Today, more than 25 years after, this 
conjecture remains open. But if indeed it is true then, at high resolution, the optimal 
entropy-constrained quantizer is a tessellating quantizer independent of the source 
distribution (as long as it is smooth). 

The distortion at high resolution of an entropy-constrained lattice vector quantizer 
is given by [44, 86] 

i^L « G(A)z/2A, (3.17) 
where v (the volume of a fundamental region of the lattice) is given by 

^2/L ^ 22(''(^)--f^)/-f'. (3.18) 

Thus, if we assume that Gersho's conjecture is true and furthermore assume that a 
lattice vector quantizer is optimal then ( 13.17b implies that ~ G(A). By inserting 
(13.18b in ( 13.17b the discrete entropy (again at high resolution) is found to be given by 

H{£) « h{X) - ^ log, (^^) [bit]. (3.19) 

The Shannon lower bound is the most widely used tool to relate the performance 
of lattice quantizers to the rate-distortion function of a source. For example, at high 
resolution, the Shannon lower bound is tight for all smooth sources, thus 

R{D) w h{X) ~ i \og,(2T:eD) [bit/dim.], (3.20) 

so that the asymptotic rate-redundancy i?red of a lattice vector quantizer over the rate- 
distortion function of a smooth source under the MSB distortion measure ij*^ 

= i log2(27reG(A)) [bit/dim.] (3.21) 

From ( 13.17b it may be noticed that the distortion of a lattice vector quantizer is 
source independent and in fact, for fixed v, the distortion only depends upon G(A). 
Furthermore, G(A) is scale and rotation invariant and depends only upon the shape of 
the fundamental region Vb of the lattice A [22]. In general the more sphere-like shape 
of Vb the smaller G(A) [22]. It follows that G(A) is lower bounded by G(S'l) the 
dimensionless normalized second moment of an L-sphere where [22] 

1 ^(L 



"For the case of resolution-constrained quantization botfi a\ and are known [58]. 
'^iJred is in fact the divergence of the quantization noise from Gaussianity in high resolution lattice vector 
quantization. 
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and where r(-) is the Gamma function. For L ^ oo we have G{Soo) — l/27re. 
The L-fold Cartesian product of the integers form an i-dimensional lattice = 
which has a hypercubic fundamental region. It can easily be computed that G{Z^) = 
1/12 which is in fact the largest dimensionless normalized second moment over all 
admissible fundamental regions [22]. Thus, 

1/12 > G(A) > G{Sl) > x^, (3.23) 

lire 

where the first two inequalities become equalities for L = 1 since in one dimension the 
only possible lattice is Z^, the scalar uniform lattice, and G{Z^) ~ G{Si) = 1/12. 
For 1 < L < oo i-spheres do not pack the Euclidean space and are therefore not 
admissible fundamental regions [22]. However, for i — oo and with a proper choice 
of lattice it is known that G(A) -> G{Sco) [161]. Table ITT] shows G{A) for the best 
known L-dimensional lattices with respect to quantization. 



Lattice name 


Dimension 


Notation 


G(A) 


G{Sl) 


Scalar 


1 




0.0833 


0.0833 


Hexagonal 


2 




0.0802 


0.0796 


BCC 


3 


As 


0.0787 


0.0770 


Schlafli 


4 


Da 


0.0766 


0.0750 




5 


D5 


0.0756 


0.0735 




6 


Eq 


0.0743 


0.0723 




7 


E7 


0.0731 


0.0713 


Gosset 


8 


Es 


0.0717 


0.0705 


Coxeter-Todd 


12 


K12 


0.0701 


0.0681 


Barnes-Walls 


16 


BWie 


0.0683 


0.0666 


Leech 


24 


A24 


0.0658 


0.0647 


Poltvrev'^ 


oo 


Aoo 


0.0585 


0.0585 



Table 3.1: The dimensionless normalized second moments of the lattice A and the L-sphere are 
denoted G'(A) and G{Sl), respectively. All figures are obtained from [22]. 

While all the lattices in Table 13.11 are the best known lattices for quantization in 
their dimensions it is in fact only Z^, ^2, ^3 and Aoo which are known to be optimal 
among all lattices [22] and furthermore, only Z^ and Aoo are known to be optimal 
among all entropy-constrained vector quantizers. 

It is interesting to compare the optimal performance of an entropy-constrained 
scalar quantizer Z^ to that of an optimal entropy-constrained infinite-dimensional lat- 
tice vector quantizer Aoo. From Table [TT] it can be seen that the rate loss i?Loss (at 

"The fact that there actually exist lattices in infinite dimensions which are capable of achieving the 
dimensionless normalized second moment of a sphere was proven in [161], a proof which was contributed 
by G. Poltyrev. 
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high resolution or at any resolution for the uniform density) when using instead of 
Aoo is given by 



^Loss — 2 



f G{Z^) 



0.2546 bit/dim. 



(3.24) 



or equivalently the increase in distortion (also known as the space-filling loss or space- 
filling gain when reversed) for using instead of K^o is given by 



lOlogi 



G{Z') 
G(Aoo) 



= 1.5329 dB. 



(3.25) 



Fig. l3.2l illustrates the space-filling loss for the lattices of Table I3TTI For compari- 
son we also show the space-filling loss of L-dimensional "quantizers" having spherical 
Voronoi cells which is given by Dloss — 101ogj^Q(G'(S'L)/G(Aoo)). 




10 12 14 16 18 20 22 24 
Dimension (L) 



Figure 3.2: Space-filling loss fi)r the lattices ofTable \3.1\ The solid line describes the space- 
filling loss of L-spheres. 



3.4 Entropy Coding 

In the previous section we saw that for stationary sources one achieves space-filling 
gains if vector quantizers are used instead of scalar quantizers. The space-filling gain 
is independent of the statistical properties of the source. In other words, whether 
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the source is i.i.d. or has memory the space-filling gain remains the same. However, 
for this to be true, we implicitly assume that any statistical redundancy (correlation) 
which might be present in the quantized signal is removed by a (lossless) entropy 
coder. Recall that the discrete entropy H{^) of the quantizer is given by ( 13.19b and 
that P{xi) denotes the probability of the symbol Xi where Xi S Assume now that 
a codeword (of the entropy coder) of length U is assigned to the symbol Xi. Then the 
average codeword length s is given by 

s = ^P(x,);,. (3.26) 

The idea of an entropy coder is to assign short codewords to very probable symbols 
and long codewords to less probable symbols in order to drive s towards its minimum. 
Since we require the (entropy) code to be lossless it means that the code should be a 
uniquely decodable code. Due to a result of Shannon we can lower bound s by the 
following theorem. 

Theorem 3.4.1. [121] The average codeword length s of a uniquely decodable binary 
code satisfies 

s>H{£). (3.27) 

In the same paper Shannon also gave an upper bound on s, i.e. s < H{^) + 1 
and he furthermore showed that if a sequence of, say s, symbols is jointly coded then 
the average number of bits per symbol satisfy 

H(£) <s < H(£) + -, (3.28) 
s 

which shows that the entropy H{^) can be approximated arbitrarily closely by enco- 
ding sufficiently long sequences [121]. 

In ( 13.261 1 we have |^| = \^\ and we thereby implicitly restrict ^ to be a dis- 
crete alphabet be it finite or countably finite, but we do in fact not always require 
that < oo. For example it is known that an entropy-constrained vector quan- 
tizer (ECVQ) may be recast in a Lagrangian sense [19,59,63] and that a Lagrangian- 
optimaj^ ECVQ always exists under general conditions on the source and distortion 
measure [63]. Furthermore, Gyorgy et al. [64] showed that, for the squared error dis- 
tortion measure, a Lagrangian-optimal ECVQ has only a finite number of codewords 
if the tail of the source distribution is lighter than the tail of the Gaussian distribution 
(of equal variance), while if the tail is heavier than that of the Gaussian distribution the 
Lagrangian-optimal ECVQ has an infinite number of codewords [64] . If the source 
distribution is Gaussian then the finiteness of the codebook depends upon the rate of 

^"The operational distortion-rate function is the infimum of the set of distortion-rate functions that can be 
obtained by use of any vector quantizer which satisfies the given entropy constraints. A Lagrangian-optimal 
ECVQ achieves points on the lower convex hull of the operational distortion-rate function and in general 
any point can be achieved by use of time-sharing [63]. 
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the codebook. In addition they also showed that for source distributions with bounded 
support the Lagrangian-optimal ECVQ has a finite number of codewordsPlFl 

In this work we will not delve into the theory of entropy coding but merely as- 
sume that there exist entropy coders which are complex enough so that (at least in 
theory) the discrete entropies of the quantizers can be reached. For more information 
about entropy coding we refer the reader to Chapter 9 of the text book by Gersho and 
Gray [45] as well as the references cited in this section. 



^' These quantizers are not unique. For example it was sliown by Gray et al. [58] tliat for tlie uniform 
density on the unit cube there exists Lagrangian-optimal ECVQs with codebooks of infinite cardinality. 
^^In Chapter|2]we will show that, in certain important cases, the cardinality of lattice codebooks is finite. 



Chapter 



Multiple-Description 
Rate-Distortion Theory 

The MD problem is concerned with lossy encoding of information for transmission 
over an unreliable i^-channel communication system. The channels may break down 
resulting in erasures and a potential loss of information at the receiving side. Which of 
the 2^ — 1 non-trivial subsets of the K channels that are working is assumed known 
at the receiving side but not at the encoder. The problem is then to design an MD 
system which, for given channel rates or a given sum rate, minimizes the distortions 
due to reconstruction of the source using information from any subsets of the channels. 
The compound channel (or composite channel) containing the K subchannels is often 
described as a packet-switched network where individual packets are either received 
errorless or not at all. In such situations the entire system is identified as a multiple- 
description system having K descriptions. 

The classical case involves two descriptions as shown in Fig. 14.11 The total rate 
Rt, also known as the sum rate, is split between the two descriptions, i.e. Rt — 
Ra + and the distortion observed at the receiver depends on which descriptions 
arrive. If both descriptions are received, the distortion (Dc) is lower than if only 
a single description is received (Dq or Di). The general JsT-channel MD problem 
involves K descriptions and is depicted in Fig. 14.21 

4.1 Information Theoretic MD Bounds 

From an information theoretic perspective the MD problem is partly about describing 
the achievable MD rate-distortion region and partly about designing good practical 
codes whose performance is (in some sense) near optimum. Before presenting the 
known information theoretic bounds we need the following definitions. 
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Figure 4.1: The traditional two-channel MD system. 
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Figure 4.2: General K-channel MD system. Descriptions are encoded at an entropy of Ri, 
i — . . . , K — 1. The erasure channel either transmits the i^^ description errorlessly or not 
at all. 



Definition 4.1.1. The MD rate-distortion region given a source and a fidelity criterion 
is the closure of the set of simultaneously achievable rates and distortions. 

Example 4.1.1. In the two-channel case the MD rate-distortion region is the closure 
of the set of achievable quintuples (i?o, ^i, -De, ^'o, -Di)- 

Definition 4.1.2. An inner bound to the MD problem is a set of achievable rate- 
distortion points for a specific source and fidelity criterion. 

Definition 4.1.3. An outer bound to the MD problem is a set of rate-distortion points, 
for a specific source and fidelity criterion, for which it is known that no points outside 
this bound can be reached. 

Definition 4.1.4. If the inner and outer bounds coincide they are called tight. 

Example 4. 1 .2. An example of inner and outer bounds for the set of achievable rate 
pairs (i?o, ^i) given some fixed distortion triple {Dc, Do, Di) is shown in Fig. 14.31 
In this example there exists a region where the inner and outer bounds meet (coincide) 
and the bounds are said to be tight within that region. 

Remark 4.1.1. The SD rate-distortion bounds form simple outer bounds to the MD 
problem. For example Ri > R{Di), i — 0, . . . , if — 1 and — R{Dc) 

where R{-) describes the SD rate-distortion function. 
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Ri A 




Ro 



Figure 4.3: The solid thin curve show an example of an outer bound and the dashed curve 
illustrates an inner bound. In the region where the bounds coincide (thick line), the bounds are 
tight. 



Definition 4.1.5. The term no excess marginal rates refers to the situation where, for 
fixed side distortions Di, the side description rates of an MD system meet the SD rate- 
distortion bounds, i.e. Ri = R{Di). At the other extreme we have the situation of no 
excess sum rate where for a given sum rate the the central distortion Dc achieves its 
minimum so that J2i=o^ — R{Dc)- 

An interestingsubset of the MD rate-distortion region is the symmetric MD rate- 
distortion region|3 The term symmetric relates to the situation where all channel 
rates (description rates) are equal and the distortion depends only upon the number of 
working channels (received descriptions) and as such not on which of the channels are 
working. This is in contrast to the asymmetric case where the description rates as well 
as side distortions are allowed to be unequal. 

Another important subset of the MD rate-distortion region is the high resolution 
region which refers to an MD rate-distortion region that becomes achievable asymp- 
totically as the description rates of the system become large relative to the variance of 
the source (or equivalently, asymptotically as the distortions tend to zero). 

4.1.1 Two- Channel Rate-Distortion Results 

El Gamal and Cover [42] obtained inner bounds to the two-channel MD problem 
(known as the EGC region) and Ozarow [107] showed that these inner bounds are 



The lower bound of this symmetric region is the symmetric MD rate-distortion function of the source. 
With a slight abuse of notation we sometimes call the MD rate-distortion function a region. 
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tight for the memoryless Gaussian source under the squared-error fidehty criterionP^ 
Ahlswede [2] and Zhang and Berger [163] showed that the EGC region is also tight for 
general sources and distortion measures in the no excess sum rate case. However, in 
the excess sum rate case it was shown by Zhang and Berger [163] that the EGC region 
is not always tight for the binary memoryless source under the Hamming distortion 
measure. Outer bounds for the binary symmetric source and Hamming distortion have 
also been obtained by Wolf, Wyner and Ziv [153], Witsenhausen [152] and Zhang and 
Berger [164]. Zamir [157, 158] obtained inner and outer bounds for smooth stationary 
sources and the squared-error fidelity criterion and further showed that the bounds be- 
come tight at high resolution. High resolution bounds for smooth sources and locally 
quadratic distortion measures have been obtained by Linder et al. [84]. Outer bounds 
for arbitrary memoryless sources and squared-error distortion measure were obtained 
by Feng and Effros [35] and Lastras-Montano and Castelli [79]. 

To summarize, the achievable MD rate-distortion region is only completely known 
for the case of two channels, squared-error fidelity criterion and the memoryless Gaus- 
sian source [42, 107]. This region consists of the convex hull of the set of achievable 
quintuples {Rq, Ri, Dq, Di, Dc) where the rates satisfy [18, 107] 

Ro>R{D^) = \^og^{^^ (4.1) 
Ri>R{D,) = \\ogJ'^\ (4.2) 



2 ^V^i, 

Ro + Ri> R{Dc) + \ log2 5{Da, Di,D,) (4.3) 
= \ log2 (^) + I log2 SiDo, D,,D,), (4.4) 



where cr^ denotes the source variance and d{-) is given by [18] 



Dc< Do + Di- a 



6{Do, Di,D,) = { (i^ + - ^) (4.5) 



, .-t .O.W.. 

(al-D,Y-y(a\-D„){al-Di)-^(D„-D^)(D,-D^)) 

and the distortions satisfy [107] 

Do > (t|2"2^" (4.6) 
Di > cri2"2-R.i (4 7) 



^2.2-2(fio+i?^l) 

72 ' 



Dc > — (4.8) 



^''it is customary in the literature to refer to the case of a memoryless Gaussian source and squared-error 
fidelity criterion as the quadratic Gaussian case. 
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where n = (1 - Do/(7j^){l - Di/aj^) and A = {DoDi/crj^) - 2-2(«o+«i). In 
general it is only possible to simultaneously achieve equality in two of the three rate 
inequalities given by (14.1b - ( I4.3l l. However, in the high side distortion case, i.e. when 
6{-) ~ 1, it is in fact possible to have equality in all three [42]. 

Fig. l4.4l shows the central distortion ( 14.8b as a function of the side distortion ( 14.6b 
in a symmetric setup where Dq = Di and Rq = Ri — 1 bit/dim. for the unit-variance 
Gaussian source. Notice that at one extreme we have optimal side distortion, i.e. 
Dq = D{Rq) = —6.02 dB, which is on the single-channel rate-distortion function of 
a unit-variance Gaussian source at 1 bit/dim. At the other extreme we have optimal 
central distortion, i.e. Dc = D{2Rq) — —12.04 dB, which is on the single-channel 
rate-distortion function of the Gaussian source at 2 bit/dim. Thus, in this example, 
the single-channel rate-distortion bounds become effective for the two-channel MD 
problem only at two extreme points. 




-6 -5.5 -5 -4.5 -4 -3.5 -3 

Side distortion [dB] 



Figure 4.4: Central distortion (Dc) as a function of side distortion (Do ~ Di) in a symmetric 
setup where Ro = Ri — 1 hit/dim. for the unit-variance Gaussian source and MSE. 

The rate region comprising the set of achievable rate pairs {Rq, Ri) which satisfy 
(14.1b . (14.2b and (14.3b is illustrated in Fig. 14.51 In this example we assume a unit- 
variance Gaussian source and choose distortions Dq = ^, Di = j and Dc = jjg. 
Notice that Rq and i?i are lower bounded by 0.5 and 1 bit/dim., respectively, and the 
sum rate is lower bounded by i?o + ^^i > 2 bit/dim. 

Ozarow's Double-Brancli Test Cliannel 

Ozarow [107] showed that the double-branch test channel depicted in Fig. |4.6| achieves 
the complete two-channel MD rate-distortion region in the quadratic Gaussian case. 
This channel has two additive noise branches Yq = X + Nq and Yi ~ X + Ni, where 
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Figure 4.5: Achievable rate pairs {Ro,Ri) for the distortion triplet {Dc, Do, Di) — 
( j^-g , ^, j) in the quadratic Gaussian case. 



all variables are Gaussian distributed and the noise pair (Nq, Ni) is independent of 
X but jointly Gaussian and negatively correlated (except from the case of no-excess 
marginal rates, in which case the noises are independent). In the symmetric case and 
when the correlation between Nq and A^i is high, i.e. near —1, the central distortion is 
close to optimum but the side distortions are then generally poor. On the other hand, 
when the side distortions are optimal, the noise pair becomes independent and the 
central distortion is not much better than either of the side distortions. The post filters 
(Wiener filters) and bi,i ~ 0, 1 describe the scalar weights which are required for 
minimum MSE (MMSE) estimation of X based on either Yq, Yi or both. At high res- 
olution this test channel is particularly simple since the filters degenerate. Specifically, 
in the symmetric case, where the noise variances are equal, the side reconstructions 
Xq and Xi become Xo = Yq and Xi — Yi, while the central reconstruction Xc 
becomes a simple average, i.e. X^ = {Xq + Xi)/2. 

Rate-Redundancy Region 

The redundancy rate-distortion function (RRD) introduced in [97] for the symmetric 
case and further developed in [52, 149] describes how fast the side distortion decays 
with increasing rate redundancy R*^^ when the central distortion Dc is fixed|^ Let 
Rc = R{Dc) be the rate needed for an SD system to achieve the (central) distortion 

^^The rate redundancy R^^^ is sometimes referred to as the excess sum rate. 
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X 




Figure 4.6: The MD optimum test channel of Ozarow [ 107]. At high resolution the filters 
degenerate so in the symmetric case we have = 1 and bi — 1/2, i — 1,2 so that Xq — 
Yo,Xi = Yi and = \{Xo + Xi). 



Dc- Then consider a symmetric setup where Dq = Di and i?o = ^^i and define 
= 2B.0 — Rc, i.e. R*^^ describes the additional rate needed for an MD system 
over that of an SD system to achieve the central distortion Dc- In order to reduce 
the side distortion Do while keeping Dc fixed it is necessary to introduce redundancy 
such that 2i?o > Rc- For a given Rc (or equivalently a given Dc) and a given R*^^ the 
side distortion for the unit-variance Gaussian source is lower bounded by [52] 




red — red 
-^red > ^red 



(4.9) 



where R*^^ = i?c — 1 + log2(l + 2^^^"). If R*^^ = we have optimum central 
distortion, i.e. no excess sum rate, but the side distortions will then generally be high. 
As we increase the rate while keeping the central distortion fixed we are able to lower 
the side distortions. Fig. 14.71 shows the side distortion Dq — Di as a function of 
the rate redundancy R*^^ when the central distortion is fixed at Dc = 2"^^"=, Rc G 
{0.5, 1, 1.5, 2, 2.5}. It is interesting to observe that when Dc is optimal, i.e. when 
-^red ~ 0, then the gap from Di to D{Ro) increases with increasing Rc- To see 
this, notice that when R*^^ = it follows from the first bound of ( |4.9t that Dq > 
(1 + 2^^^')/2- The second bound of (14.9b is actually the SD rate-distortion bound, 
i.e. 2^'^^'^+^";'i^ — 2~^^°, and the gap between these two bounds is shown in Table l4. l] 



Two-Channel High-Resolution Results 



Based on the results for the Gaussian source of Ozarow [107] it was shown by 
Vaishampayan et al. [136, 137] that at high resolution and for the symmetric case. 
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Figure 4.7: Side distortions Do = Di as a function of rate redundancy R%j. For each curve 
the central distortion is held fixed at Dc ~ 2~^^''. The circles mark the points beyond which 
the second bound of \4.9\ becomes effective. This example is from [52]. 



if the side distortions satisfy 

Do = a%b2-'^"°^^-''\ (4.10) 
for < a < 1 and 5 > 1 then the central distortion is lower bounded by 

£)^> ^2-2«o(l+a)^ (4 11) 

46 

which leads to a simple bound on the distortion product DcDq, that is 

4 

DcDo>^2-^^°. (4.12) 

It was further shown that an optimal two-channel scheme achieves equality in (14.11b 
and therefore also in ( 14.12b at high resolution and when Dc ^ Dq. Since (14.12b 
is independent of a it serves as a simple means of relating the performance of MD 
schemes to the information theoretic rate-distortion bounds of [107]. It is therefore a 
standard figure of merit when assessing the performance of two-channel MD schemes 
at high resolution. For small ratios of Dq/D^ it is not possible to achieve equality 
in ( 14.121 ). However, at high resolution the more general but less used distortion product 
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Rc 


i?o 


(l + 2-2flc)/2 




Gap 


0.5 


0.25 


0.75 


0.707 


0.043 


1 


0.5 


0.625 


0.5 


0.125 


1.5 


0.75 


0.563 


0.354 


0.209 


2 


1 


0.531 


0.25 


0.281 


2.5 


1.25 


0.516 


0.177 


0.339 



Table 4.1: The gap between the two bounds of i4. 91 ) when i?,*^ = 0. In this case Ro — Rc/2. 



is also achievable [137] 

^^^°^XT3lb^2--o, (4.13) 

which meets the lower bound of ( 14.121 ) if Dc/Dq 0. If Dq is optimal, i.e. if 
Do = D{Rq), then it follows from [107] that Dc > Dq/2. Using the ratio Dc/Dq = 

4 

1/2 in (14.131 1 yields DcDq ~ Z^2^^^" which is twice as large as the lower bound 
of (021 1. 

Fig- SS] compares the high resolution approximations given by (14.10b and (14.11b 
(solid lines) to the true bounds given by (14.6b and (14.8b (dashed lines) for the case of 
a unit-variance memoryless Gaussian source and 6 = 1. Notice that the asymptotic 
expressions meet the true bounds within a growing interval as the rate increases. Since 
a is positively bounded away from zero and always less than one, the interval where 
they meet will never include the entire high resolution region. For example only large 



distortion ratios Dq /Dc are achievable]^ 

The asymmetric situation is often neglected but it is in fact fairly simple to come 
up with a distortion product in the spirit of ( 14.12b . Let us first rewrite the central and 
side distortions in Ozarow's solution by use of the entropy power Px as was done by 
Zamir [157, 158], that is 

D,>Px2-^^', 1 = 0,1, (4.14) 

and 

Py2-2(i?o + fli) 

Dc > n, (4.15) 

- i-(|Vn-VA|+)^' 

where 

U = {l-Do/Px)il-Di/Px) (4.16) 

and 

A = DqDi/P^ -2-^'-^°+'^^\ (4.17) 



^''In Section lTTl Remark l?. 1 . 1 I we explain in more detail why the asymptotic curves never meet the true 
curves at the extreme points. 
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Figure 4.8: The central distortion Dc as a function of side distortions Do — D\ at different 
rates Rq = Ri G {1, . . . , 5}. The dashed lines illustrate the true distortion bounds given 
by \4.6\ and \4.8^ and the solid lines represent the high resolution asymptotic bounds given 
by (|?JOl l and Ii4l7\ . 



and wher^ 

, , , . I x, if a; > 

/ , . (4.18) 
10, otherwise. 

An advantage of Zamir's solution is that it acts as an outer bound to the MD prob- 
lem for general sources under the squared-error distortion measure. For the memo- 
ryless Gaussian source it becomes tight at any resolution, i.e. it becomes identical to 
Ozarow's solution, and for arbitrary smooth stationary sources it becomes asymptoti- 
cally tight at high resolution. 

Lemma 4.1.1. if 2-2(^^0+^1) ^ jj^jj^ < A, « = 0, 1 then 

Proof. Let us expand the denominator in ( 14.151 ) aj^ 

1 - (Vn- v^)' = 1 - (V{l~Do/Px){l-Di/Px) 



I ■ 1+ becomes effective only in the high side distortion case, i.e. when Dq + Di > o"^(l 
2-2{flo+-Ri)) [158]. 

^*Here we neglect the high side distortion case. 
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1 - ((1 - Do/Px){l - D,/Px) + DoD,/P^ - 2-2(«o+i^i) 
- 2^(1 - Do/Px)a - D,/Px){D^D,/Pl - 2-2(%+fl.i))) 
Do/Px + DjPx - 2DoDi/P^ + 2-2(flo+fli) 
+ 2(DoD^/PI - DlDi/P^x - D^Dl/Pl + {D^D^/Plf 

^ (1 - Do/Px - Di/Px + i^o^i/i"l)2~'(^«+^^ 
D0 + D1+ 2VA^ 



Px 



(4.20) 



where the approximation follows from the assumption of high resolution, i.e. Ri 

00, i = 0, 1, sothatwehave2-2(«o+«i) < DqDi < A- The inequality 2^2(i?o+fli) 
^ DqDi is valid when we have excess marginal rates, i.e. when at least one of the 
side decoders is not operating on its lower bound. As such we assume that Di grows 
as ff{2^^^') where Ri < Ri and ^0 + Ri < Rq + Ri and it follows that the 
entire expression is dominated by terms that grow as 1^(2^^^') or ^(2^^-'^°+^i^). 
Inserting (I4.20l i into ( 14.15b leads to 



P^2-2(flo+-Rl) 
Dr > 



(4.21) 



p2 

^X _2-2(i7,o+Ki 



D0 + D1 + 

which completes the proof since Px = 2^''*^"^'' /(27re). □ 

Remark 4.1.2. It follows that an optimal asymmetric (or symmetric) MD system 
achieves equality in ( |4.19t at high resolution for arbitrary (smooth) sources. Notice 
that the bound ( 14.191 ) holds for arbitrary bit distributions of i?o and Ri as long as their 
sum remains constant and the inequalities ( 14.141 ) and ( 14.15b are satisfied (or more cor- 
rectly that the corresponding lower bounds on the individual side rates and their sum 
rate are satisfied). 



4.1.2 iT-Channel Rate-Distortion Results 

Recently, an achievable -channel MD rate-distortion region was obtained by Venka- 
taramani, Kramer and Goyal [141, 142] for arbitrary memoryless sources and single- 
letter distortion measures. This region generally takes a complicated form but in the 
quadratic Gaussian case it becomes simpler. The region presented in [142] describes 
an asymmetric MD rate-distortion region and includes as a special case the symmetric 
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MD rate-distortion region. The construction of this region rehes upon forming lay- 
ers of conditional random codebooks. It was, however, observed by Pradhan, Puri 
and Ramchandran in a series of papers [109-1 14] that by exploiting recent results on 
distributed source coding it is possible to replace the conditional codebooks with uni- 
versal codebooks whereby the codebookrate can be reduced through random binning. 
While Pradhan et al. limited their interests to the symmetric case it can be shown that 
their results carry over to the asymmetric case as well. This has recently been done 
by Wang and Viswanath [146] who further extended the results to the case of vector 
Gaussian sources and covariance distortion measure constraints. 

The largest known achievable rate-distortion region for the iC-channel MD prob- 
lem is that of Pradhan et al. [111,114]E1 Common for all the achievable rate-distortion 
regions is that they represent inner bounds and it is currently not known whether they 
can be further improved. However, for the quadratic Gaussian case it was conjectured 
in [114] that their bound is in fact tight. That conjecture remains open. 

The key ideas behind the achievable region obtained by Pradhan et al. are well 
explained in [111, 114] and we will here repeat some of their insights and results 
before presenting the largest known iiT-channel achievable rate-distortion region. 

Consider a packet-erasure channel with parameters K and k, i.e. at least k out of 
K descriptions are received. For the moment being, we assume k — 1. Generate 
K independent random codebooks, say ■ ■ • each of rate R. The source 

is now separately and independently quantized using each of the codebooks. The 
index of the nearest codeword in the i*'* codebook is transmitted on the channel. 
A code constructed in this way was dubbed a source-channel erasure code in [111] 
which we, for notational convenience, abridge to {K, k) SCEC. Notice that since 
each of the individual codebooks are optimal for the source then if only a single index 
is received the source is reconstructed with a distortion that is on the distortion-rate 
function D{R) of the source. However, if more than one index is received, the quality 
of the reconstructed signal can be strictly improved due to multiple versions of the 
quantized source. The above scheme is generalized to {K, k) SCEC for fc > 1 by 
making use of random binning. This is possible since the quantized variables are 
assumed (symmetrically) correlated so that general results of distributed source coding 
are applicable. Specifically, due to celebrated results of Slepian and Wolf [124] and 
Wyner and Ziv [154], if it is assumed that some k out of the set of K correlated 
variables are received then (by e.g. use of random binning) it is possible to encode at 
a rate close to the joint entropy of any k variables, in a distributed fashion, so that the 
encoder does not need to know which k variables that are received. It is usually then 
not possible to decode on reception of fewer than k variables. 

Before presenting the main theorem of [1 1 1] which describes the achievable rate- 
distortion regions of [K, k) SCECs in the general case of\<k<K, we need some 
definitions. 



Outer bounds for the X-channel quadratic Gaussian problem were presented in [142]. 
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Definition 4.1.6. D^^^^^ denotes the distortion when receiving k out of K descrip- 
tions. 

Definition 4.1.7. A tuple (i?, D^^'''^ , D(^^'=+i) , . . . , Dl^^-^^) ) is said to be achievable 
if for arbitrary 5 > 0, there exists, for sufficiently large block length L, a {K, k) SCEC 
with parameters (L, 6, A^, A^+i, . . . , A^) with 

e < 2^^-^'+^^ and A,, < D^^'^^ +6, h = k,k + I, . . . ,K. (4.22) 

Let = {/ : / C {0, . . . , X - 1}, |/| > k}. A {K, k) SCEC with parameters 
(L, 0, Afc, Afc+i, . . . , /S.k) is defined by a set of K encoding functions [111] 

Fii ^-^{l,2,...,e}, i = Q,...,K^l, (4.23) 

and a set of | ,fk \ decoding functions 

G/: (g){l,2,...,e}-^ J^, V/e^fc (4.24) 

where denotes the Cartesian product and for all /i € {fc, fc+1, . . . , Jf} and X £ 2^ 
we have 

A;, - G,(i^n (^), • • • , F,. (X))) 

Theorem 4.1.1 ( [1 11], Th. 1). For a probability distributioi0 

p{x,yQ, ■ ■ ■,yK-i) = q{x)p{yo, ■ ■ -^vk-iIx) (4.26) 

defined over ^ where ^ is some finite alphabet, p{yQ, . . . , yK-i\x) is sym- 
metric, and a set of decoding functions V/ € J^k, 9i ■ ^ , if 

Ep(X,gi{Yi)) < V/ e A- (4.27) 

and 

R>^H{Yo,...,Yk-,)- ^H{Yo,...,Yk-i\X) (4.28) 

then {R, i:)(^^*=) , D(.K,k+i) ^ ^ £)(/^,-ff)) jg ^n achievable rate-distortion tuple. 

In [114] an achievable rate-distortion region for the X-channel MD problem was 
presented. The region was obtained by constructing a number of layers within each 
description where the set of all j*'* layers across the K descriptions corresponds to 
a {K,j) SCEC. Assume that it is desired to achieve some distortion triplet (D^^^^^, 
£)(3:2) £)(3,3)-j Jqj. ^ three-channel system. Then first a (3,1) SCEC is constructed 
using a rate of R'^^^ bit/dim. per description. We use the superscript to distinguish 

'"To avoid clutter we omit the subscripts on tlie probability distributions in this section. 
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between the rate Rq of encoder in an asymmetric setup and the rate R^*^^ of layer 
in a symmetric setup. The rate i?^'^^ is chosen such that q^^x be achieved with 

the reception of any single description. If two descriptions are received the distortion 
is further decreased. However if 13(3,2) achieved on the reception of any two 

descriptions, then a (3,2) SCEC is constructed at a rate i?^^^ bit/dim. per description. 
The rate i?'^' is chosen such that 13(3,2) ^^j^ achieved on the reception of any two 
descriptions. If £)(3 3) jg jjqj achieved on the reception of all three descriptions, a 
refinement layer is constructed at a rate of i?(^^ bit/dim. per description, see Fig. 14.91 
Each description contains three layers, e.g. description i consists of a concatenation 
of Loi, Lii and L2i- It is important to see that the first layer, i.e. the (3,1) SCEC is 
constructed exactly as described by Theorem 14. 1.11 The second layer, i.e. the (3,2) 
SCEC differs from the construction in that of the binning rate. Since, on reception 
of any two descriptions (say description and 1), we have not only the two second 
layers {Liq and Ln) but also two base layers (Lqo and Lqi). This makes it possible to 
decrease the binning rate i?(^) by exploiting correlation across descriptions as well as 
across layers. The final layer can be a simple refinement layer where bits are evenly 
split among the three descriptions or for example a (3,3) SCEC. The rate of each 
description is then given by i? = i?(°) + i?(^) + i?(^) and the total rate is 3i?. 



Description 
Description 1 
Description 2 



= (0) 



ill 



il2 



L2I 



(3, 1) SCEC (3, 2) SCEC (3, 3) SCEC 



Figure 4.9: Concatenation of (3,1), (3,2) and (3,3) SCECs to achieve the distortion triplet 
^ £)(3,2) ^ £)(3,3) -J description contains three layers and the rate of each description 

isR = R^''^ +R^^> +Rf''K 



We are now in a position to introduce the main theorem of [1 14] which describes 
an achievable rate-distortion region for the concatenations of {K, k) SCECs. Let Yij 
be a random variable in the i"* layer and j*'' description and let I^^^ = {0, . . . , K — 
1}. For i £ Iq^^^, let ^i/^'-i = O^io, Yn, ■ ■ ■ , Ym-i) represent K random variables 
in the i*'* layer taking values in alphabet Let Yk-i be the last layer refinement 
variable taking values in the alphabet ^^k-i and 

^lo'^Io'' " (^o/(f-i'^i/(f-i' • • ■ '^(/f-2)/o'^-0- (4-29) 
A joint distribution p{yjK-2jK-i,yK^i\x) is called symmetric if for all 1 < < 



Section 4.1 



Information Theoretic MD Bounds 



51 



K where i e the following is true: the joint distribution of Yk-i and all 

(ro + ri + • • • + rK-2) random variables where any r,; are chosen from the i*^ layer, 
conditioned on x, is the same. 

Theorem 4.1.2 ( [1 14], Th. 2). For any probabiUty distribution 

p{x,yiK-2jK-i,yK-i) ^ p{x)p{yjK-2jK-i,yK-i\x) (4.30) 

where p{y jk-i , yK-i\x) is symmetric, defined over x "S^q^ x x • • • x 
'S^K-i ^ "S^K-i, and a set of decoding functions given 



gi ■ ^TqI'I X ... X ^IjLi ^ ^ V/ C /o^-i 



(4.31) 



: X X ... X ^/_2 X ^if-i ^ 
the convex closure of (i?, D^-^'^\ D'^^'^\ . . . , is achievable where 

i?p/(X,g/(y^i.i-i^))<I?(^-l^l) V/c/o'^-\ (4.32) 

EpjK-^(X,gj.-^{Y,K-2,K-.,YK-i)) < (4.33) 

and 

K-l 

R > HiYoo) + T^(^fe-i/r^l^/^-/o-0 

k=2 (4.34) 

Ji I Jo -"o ' ^ -"o -'0 

The main difference between Theorem 14. 1 . 11 and Theorem 14. 1.21 is that the latter 
theorem considers the complete i^-tuple of distortions , ^ . . . ^ ]j(k,k) 

whereas the former theorem considers the {K — k + l)-tuple of distortions (D(^''=) , 
]jiK,k+i) ^ ^ ^ ^jj(K,K)y Hence, an SCEC based on the construction presented 
in [111] is specifically tailored to networks where it is known that at least k chan- 
nels out K channels are always working. With the construction presented in [114] it 
is possible to concatenate several SCECs and obtain a code that works for networks 
where the number of working channels is not known a priori. 



4.1.3 Quadratic Gaussian A"-Channel Rate-Distortion Region 

We will now describe the achievable if-channel rate-distortion region for the quadratic 
Gaussian case. This appears to be the only case where explicit (and relatively simple) 
closed-form expressions for rate and distortion have been found. 

31 jjfJ denotes the j times Cartesian product of the alphabet 
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Consider a unit-variance Gaussian source X and define the random variables 

Yi,i = 0,...,K -1, given by 



Y,=X + Q, 



(4.35) 



where the Q^'s are identically distributed jointly Gaussian random variables (indepen- 
dent of X) with variance and covariance matrix Q given by 



Q = a 



1 


Pq 


Pq ■ 


• Pq 


Pi 


1 


Pq ■ 


■ Pq 


Pq 


Pq 


1 • 


■ Pq 



1 



(4.36) 



Pq Pq Pq 

where, for K > l,it is required that the correlation coefficient satisfies —1/{K— 1) < 
Pq < 1 to ensure that Q is positive semidefinite [142]. In the case of Ozarow's double 
branch test channel for K = 2 descriptions, we only need to consider non positive 
Pq's. This is, in fact, also the case for K > 2 descriptions [142]. 

It is easy to show that the MMSE when estimating X from any set of m F^'s is 
given by [111,142] 



a^Jl + {m-l)pq) 



(4.37) 



m + 0-2(1 + _ l)pg)' 

We now focus on the {K, k) SCEC as presented in Theorem 14. 1.1 1 The rate of each 
description is given by [111] 

l/fc . . . l/K 



1 k + <jf^{l + {k-l)pq) 



1 - Pg 



The quantization error variance cr^ can now be obtained from ( 14.38b 

■\^(K-\)pq\^^'' 



Cji^kUl- pq)2 



2kR 



Pq 



(4.38) 



(4.39) 



We will follow [111] and look at the performance of a {K, k) SCEC in three dif- 
ferent situations distinguished by the amount of correlation pq introduced in the quan- 
tization noise. 



Independent quantization noise: pq = 

The quantization noise is i.i.d., i.e. pq — 0, hence Q is diagonal. Assuming that the 



quantization noise is normalized such that cr^ — kj (2?^^ ~ l), we get the following 



expressions for the distortion 



jj{K.k+r) ^ ^ ^ ^ ^ for < r < - fc, (4.40) 

'q' ' ' ' ' 



ai + (k + r) 22'=-"(fc + r) -r 
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and 

jjiK^m) for < TO < /c. (4.41) 

The distortion when receiving k descriptions is optimal, i.e. D^^-*^) = 2-2fc_R 

Correlated quantization noise: pq = p* 

The amount of correlation p* needed in order to be on the distortion-rate function on 
the reception of k ^ K descriptions is given by 



P;-- .,. , , ~-7^- (4-42) 



(if - 1)22^"RTT ~ 1 

This leads to the following performance 



j^(K,r) (--^ _ 2-2^^) for 0<r<K. (4.43) 

K 

Notice that the distortion when receiving K descriptions is optimal, i.e. £)(^'^) = 

2-2KR 



Correlated quantization noise: p* < pq < 

Here a varying degree of correlation is introduced and the performance is given by 

/!^ > + (r — l)pa) 

= ^TT^T^-^V^T^ forfc <r<if, (4.44) 
a^{l + [r -l)pq) + r 

and 

jjiK^m) for < TO < /c. (4.45) 

Fig. 14.101 shows the three-channel distortion L)(3,3) ^ function of the two- 
channel distortion D^'^'^^ when varying pq and keeping the rate constant by use 
of ( |4.39l l. In this example we use a (3, 2) SCEC with R = 1 bit/dim. At one end 
we have D^^'^^ — —12.0412 dB which is on the distortion-rate function of the source 
and at the other end we have D'-^'^^ = —18.0618 dB which is also on the distortion- 
rate function. 

In Fig. 14.111 we show the simultaneously achievable one-channel _D'^^^), two- 
channel d(3.2) ^jj(J three-channel Dt-^ 3) distortions for the unit-variance memoryless 
Gaussian source at 1 bit/dim. when using a (3, 1) SCEC. It is interesting to observe that 
while it is possible to achieve optimum one-channel distortion (pq = D'^'^) sa 
-6 dB) and optimum three-channel distortion (pq ^ -1/2 ^ ui^^s) ~ _i8 JB) it 
is not possible to drive the two-channel distortion towards its optimum (\/pq, D^^'^^ < 
— 12 dB). In other words, a (3, 1) SCEC can achieve optimal one-channel and three- 
channel performance and a (3, 2) SCEC can achieve optimal two-channel and three- 
channel performance. 
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2-channel distortion [dB] 



Figure 4.10: Three-channel versus two-channel distortions for a (3,2) SCEC at R = 1 bit/dim. 
for the unit-variance Gaussian source. This example is from [111]. 



Let us now look at the achievable three-channel region presented in [1 14] for the 
the memoryless Gaussian source. Let R bit/dim. per description be the rate of trans- 
mission. Let the random variables in the three layers be defined as 

Ya, =X + Qoj , Yi, = X + Qi, , and = X + Q2 for j G , (4.46) 



where for i g /q , Q^p are symmetrically distributed Gaussian random variables with 
variance cr^. and correlation coefficient pq. and Q2 is a Gaussian random variable 
with variance (j"^^. Qqj2,Qij2 and Q2 are independent of each other and X. By 
changing the four independent variables R'-^\ R'-^\ pqg and pq-^ different trade-offs 
between D^^.i) , jji3,2) and D^^.a) 

can be made. The correlation coefficients are lower 

bounded by [114] 

The variance aq^ of the base layer follows from (14.391 1 by letting k = 1, that is 

^-2 ^ 22«''"(i + 2p,jl(l - - 1 (4.48) 

and it can be shown that [11 4] 

^-2 ^ l + Pg, _ 1 + 24«'^' (1 + pq„ + 2/a^2j(l + Pq, ^ 2pl)i 

(I + P90X 2 2(l + p,J(l-p,ji 

(4.49) 
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Figure 4.11: The simultaneously achievable one-channel, two-channel and three-channel dis- 
tortions for the unit-variance Gaussian source at 1 bit/dim. for a (3,1) SCEC. 



Let MSE2 denote the distortion given by any two base-layer random variables, and 
MSE3 denote the distortion given by all the random variables in the base and the 
second layer Hence, MSE2 denotes the MMSE obtained in estimating the source X 
using either {Yqq, Yqi), (Yqq, Y02) or (Yqi,Yq2). Similarly MSE3 denotes the MMSE 
in estimating X from (Ybo, Yqi, Y02, Yio, Yu, Y12). From ( 14.37b it follows that 



MSE2 = (4.50) 



and it can also be shown that [11 4] 



MSE, = 



(4.51) 



and 



MSE3 

26(fl-fl(0)-i?(i)) _ I ■ 



(4.52) 
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Finally, we have [1 14] 



£)'(3,1) ^ 



90 



1 



D 



'(3,2) 



£)'(3,3) ^ 



4(l + p,jMSE2 



MSE3 



(4.53) 
(4.54) 
(4.55) 



92 



The lower convex hull of (il)'(3.i) £)'(3,2) ^ ^'(3,3)-j corresponds to an achievable 
distortion tuple (Z)^^'^), Z?^^'^), See Fig. |4TT2l for an example of an achievable 

distortion region for the unit- variance memoryless Gaussian source fori?("' = = 
0.5 bit/dim. per description and a description rate of i? = + R^^^ = 1 bit/dim. In 
this plot the correlation values are varied throughout the range given by ( |4.47| l. 



-12.5- 



m 

S -13- 



-13.5- 



-14- 



-14.5- 




D'^'^) [dB] 



Figure 4.12: Achievable distortion region for R'^'^^ = J?'^' = 0.5 bit/dim. per description. The 
description rate is R = i?^"-* -|- J?'^-' — 1 bit/dim. The dense peak is in the front. 



4.2 Multiple-Description Quantization 

The previous section described known information theoretic bounds. These bounds 
were shown to be achievable by use of random codebooks. Unfortunately random 
codebooks are usually not very practical due to e.g. high search complexity and large 
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memory requirements. From a practical point of view it is therefore desirable to avoid 
random codebooks, which is the case for the MD schemes we present in this section. 

Existing MD schemes can roughly be divided into three categories: quantizer- 
based, transform-based and source-channel erasure codes based. Quantizer-based 
schemes include scalar quantization [5, 10,39,68, 130, 131, 135, 138], trellis coded 
quantization [67, 137, 147] and vector quantization [15, 17, 18,27,28,36,37,48,51, 
73, 75, 98, 99, 103-105, 120, 127, 129, 139, 155]. Transform-based approaches in- 
clude correlating transforms [49,52,53,97, 148], overcomplete expansions and fil- 
terbanks [4,20,29,54,55,76]. Schemes based on source-channel erasure codes were 
presented in [109-1 14]. For further details on many existing MD techniques we refer 
the reader to the excellent survey article by Goyal [50]. 

The work in this thesis is based on lattice vector quantization and belongs therefore 
to the first of the catagories mentioned above to which we will also restrict attention. 

4.2.1 Scalar Two- Channel Quantization with Index Assignments 

In some of the earliest MD schemes it was recognized that two separate low -resolution 
quantizers may be combined to form a high-resolution quantizer. The cells of the high- 
resolution quantizer are formed as the intersections of the cells of the low-resolution 
quantizers [50]. The two low-resolution quantizers are traditionally called the side 
quantizers and their joint quantizer, the high-resolution quantizer, is called the central 
quantizer. If the side quantizers are regular quantizers, i.e. their cells form connected 
regions, then the central quantizer is not much better than the best of the two side 
quantizers. However, if disjoint cells are allowed in the side quantizers, then a much 
better central quantizer can be formed. According to Goyal's survey article [50], the 
idea of using disjoint cells in the side quantizers seems to originate from some unpub- 
lished work of Reudink [116]. Fig |4.13(a)| shows an example where two regular side 
quantizers Qq and Qi each having 3 cells are combined to form a central quantizer Qc 
having 5 cells. Hence, the resolution of the central quantizer is only about twice that of 
either one of the two side quantizers. Fig. |4.13(b)| shows an example where one of the 
side quantizers have disjoint cells which makes it possible to achieve a very good joint 
quantizer. In this case both side quantizers have three cells but Qi has disjoint cells. 
The central quantizer has 9 cells which is equal to the product of the number of cells 
of the side quantizers. Hence, the resolution of the central quantizer is comparable to 
an optimal single description scalar quantizer operating at the sum rate of the two side 
quantizers. The price, however, is relatively poor performance of side quantizer Qi. 

The idea of using two quantizers with disjoint cells as side quantizers and their 
intersections as a central quantizer was independently discovered by Vaishampayan 
[135] some years after Reudink. Vaishampayan proposed a systematic way to control 
the redundancy in the two side quantizers by use of an index assignment matrix [135]. 
The idea is to first partition the real line into intervals in order to obtain the central 
quantizer and then assign a set of central cells to each cell in the side quantizers. For 



58 



(Chapter 4) 



Multiple-Description Rate-Distortion Theory 




(a) Regular side quantizers (b) Iiregular side quantizers 



Figure 4.13: Two quantizers are able to refine each other if their cell bounderies do not coin- 
cide. In (a) both Qo and Qi are good quantizers but Qc is poor. In (b) quantizer Qi is poor 
whereas Qo and Qc are both good. 



example let us partition the real line into 7 intervals as shown in Fig. |4.14(a)| (the 
bottom quantizer is the central quantizer). We then construct the index assignment 
matrix as shown in Fig. |4.14(b)| Each column of the matrix represent a cell of the 
side quantizer shown in the top of Fig. |4.14(a)| Since there are four columns the 
side quantizer has four cells. Similarly, the four rows of the matrix represent the four 
cells of the second side quantizer (the middle quantizer of Fig. |4.14(a)[ ). The central 
quantizer in this design, which is based on the two main diagonals and where the side 
quantizers have connected cells, is known as a staggered quantizerl^ If we only use 
the main diagonal of the index assignment matrix we get a repetition code. In this case 
the side quantizers are identical and they are therefore not able to refine each other, 
which means that the central distortion will be equal to the side distortions. 

By placing more elements (numbers) in the index assignment matrix the central 
quantizer will have more cells and the central distortion can therefore be reduced. 
There is a trade-off here, since placing more elements in the matrix will usually cause 
the cells of the side quantizers to be disjoint and the side distortion will then increase. 
Fig. |4.15(b)| shows an example where the index assignment matrix is full and the 
central quantizer therefore has 16 cells. Hence, the central distortion is minimized. 
From Fig. |4.15(a)| it is clear that the cells of the side quantizers are disjoint and since 
each cell is spread over a large region of the central quantizer the side distortion will 
be large. 

The main difficulty of the design proposed by Vaishampayan lies in finding good 
index assignments, i.e. constructing the index assignment matrix. In [135] several 
heuristic designs were proposed for the case of symmetric resolution-constrained MD 
scalar quantization. Their performance at high resolution was evaluated in [ 1 36] and it 

^^It is often possible to make the second side quantizer a translation of the first side quantizer and use 
their intersection as the central quantizer. The quality improvement of a central quantizer constructed this 
way over that of the side quantizers is known as the staggering gain. However, as first observed in [39] 
and further analyzed in [132] the staggering gain dissappears when good high dimensional lattice vector 
quantizers are used. 
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(a) Side and central quantizers 
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(b) Index assignment matrix 



Figure 4.14: (a) The two side quantizers each having four cells are offset from each other. 

Their intersection forms the central quantizer having 7 cells, (b) shows the corresponding 
index assignment matrix for the quantizers. The columns of the matrix form a side quantizer 
and the rows also form a side quantizer 



0,1, ,1,2,3,2,1,0,1,2 



2 I 3 
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(a) Side and central quantizers 
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(b) Index assignment matrix 



Figure 4.15: (a) The two side quantizers each having four cells are not identical. Their intersec- 
tion forms the central quantizer having 16 cells, (b) shows the corresponding index assignment 
matrix for the quantizers. The columns of the matrix form a side quantizer and the rows also 
form a side quantizer 
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was shown that in the quadratic Gaussian case, the distortion product DcDq was 8.69 
dB away from the optimal high resolution distortion product ( 14.12b . Vaishampayan 
and Domaszewicz [138] then proposed an entropy-constrained MD scalar quantizer 
where the index-assignment matrix was optimized using a generalized Lloyd algo- 
rithm. The distortion product of this design was shown to be only 3.06 dB away from 
the theoretical optimum [136]. Recall that the space-filling loss of an SD scalar quan- 
tizer is 1.53 dB so that, quite surprisingly, the gap to the optimal distortion product of 
a two-description scalar quantizer is twice the scalar space-filling loss. The design of 
good index assignments for the scalar case is further considered in [10]. 

It is known that the entropy-constrained scalar uniform quantizer is optimal in the 
SD case, see Chapter [3] This result, however, does not carry over to the MD case. 
Goyal et al. [51, 73] were the first to recognize that by slightly modifying the central 
quantizer in a way so that it no longer forms a lattice, it is possible to reduce the 
distortion product not only in the scalar case but also in the two-dimensional case. 
This phenomenon was further investigated by Tian et al. [129-131] who showed that 
the scalar distortion product can be further improved by 0.4 dB by modifying the 
central quantizer. 

4.2.2 Lattice Vector Quantization for IMultiple Descriptions 

Recently, Servetto, Vaishampayan and Sloane [120, 139] presented a clever construc- 
tion based on lattices, which at high resolution and asymptotically in vector dimension 
is able to achieve the symmetric two-channel MD rate-distortion region. The design 
of [120, 139] is again based on index assignments which are non-linear mappings that 
lead to a curious result. Let — Rq = Ri denote the rate of each of the side 
quantizers and let < a < 1. Then, at high resolution, the central distortion Dc 
satisfies [139] 

lim i?,22«=(i+") iG(A)22''(^), (4.56) 
whereas the side distortions Do = Di satisfy 

lim 002^^'^^^"^ = G{Sl)2'^''^^\ (4.57) 

where G{K) and G{Sl) are the dimensionless normalized second moments of the 
central lattice A and an i-sphere, respectively. Thus, remarkably, the performance 
of the side quantizers is identical to that of quantizers having spherical Voronoi cells; 
note that in the SD case this is not possible for 1 < L < oo. 

The design presented in [120, 139] is based on a central lattice A and a single 
sublattice A^ of index = |A/As|. Each central lattice point A G A is mapped to 
a pair of sublattice points (Aq, Ai) e As x As using an index assignment function. 
A pair of sublattice points is called an edge. The edges are constructed by pairing 
closely spaced (in the Euclidean sense) sublattice points. By exploiting the direction 
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of an edge (i.e. the pair (Aq, Ai) is distinguishable from (Ai, Aq)) it is possible to use 
an edge twice. In order to construct the edges as well as the assignment of edges to 
central lattice points, geometric properties of the lattices are exploited. Specifically, 
the edges (Ao, Ai) and (Ai, Ao) are mapped to the central lattice points A e A and 
A' S A which satisfy A + A' = Aq + Ai and furthermore, the distance from the midpoint 
of an edge and the associated central lattice point should be as small as possible (when 
averaged over all edges and the corresponding assigned central lattice points). Only a 
small number of edges and assignments needs to be found, whereafter the symmetry 
of the lattices can be exploited in order to cover the entire lattice. Examples of edge 
constructions and assignments are presented in [120, 139]. 

The asymmetric case was considered by Diggavi, Sloane and Vaishampayan [27, 
28] who constructed a two-channel scheme also based on index assignments and 
which, at high resolution and asymptotically in vector dimension, is able to reach 
the entire two-channel MD rate-distortion region. Specifically, at high resolution, the 
central distortion satisfies [28] 

= G(A)22(''(^)-«'^), (4.58) 
where i?c is the rate of the central quantizer and the side distortions satisfy 

Do = _^ll_G(A,)22'^(^)2"2(«°+«i-«=), (4.59) 
(70 +7i)^ 

and 

Di = , ^" g(A^)2^^W2-^(^"+^^-^-), (4.60) 
(70+71) 

where G(As) is the dimensionless normalized second moment of a sublattice A^, 
which is geometrically-similar to both side lattices Ai,i = 0, 1, and 70,71 G K"'' 

are weights which are introduced to control the asymmetry in the side distortions. 

2 

Notice that in the distortion-balanced case we have 70 ~ 71 so that ^^^^^"^^^^ — i 
and if 7o = or 71 = then the design degenerates to a successive refinement 
scheme [28, 32]. It is worth emphasizing that the side quantizers in the asymmetric 
design do generally not achieve the sphere bound in finite dimensions as was the case 
of the symmetric design. 

The design presented in [27, 28] is based on a central lattice A, two sublattices 
Ao C A and Ai C A of index iVo = |A/Ao| and A^i = |A/Ai|, respectively, and a 
product lattice Att C Ai, i = 0, 1, of index iV^r = NqNi. The Voronoi cell V7r(A7r) 
of the product lattice point A^r G A^r contains A^i sublattice points of Aq and TVo 
sublattice points of Ai, see Fig. l4.16l for an example where A^o = 5 and iVi = 9. In 
this example, only the 45 central lattice points located within (0) need to have edges 
assigned. The remaining assignments are done simply by shifting these assignments 
by A^ G A^. In other words, if the edge (Ao, Ai) is assigned to A then the edge 
(Ao + AttjAi + Att) is assigned to A + A^r. We say that the assignments are shift 
invariant with respect to the product lattice. 
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The 45 edges are constructed in the following way. First, create the set Eaq con- 
taining the nine sublattice points of Aq which are located within V,r{0), i.e. 

Ea„ = {(0, 0), (-3, 1), (-2,-1), (-1, 2), (-1, -3), (1, -2), (1, 3), (2, 1), (3, 1)}. 

Let Ao be the first element of Ea^, i.e. Ao = (0,0). Pair Ao 
with the five Ai points located within VTr{0). Thus, at this point we 
have five edges; {(0, 0), (0, 0)}, {(0, 0), (0, 3)}, {(0, 0), (0, -3)}, {(0, 0), (3, 0)} and 
{(0, 0), (—3, 0)}. Consider now the second element of Ea„, i.e. Aq = (—3, 1). Shift 
Vtt (0) so that it is centered at Ao (illustrated by the dashed square in Fig. l4.16l l. For no- 
tational convenience we denote V^(0)+Ao by V^(Ao). WenowpairAo = (— 3,l)with 
the five sublattice points of Ai which are contained within V"7r(Ao), i.e. Ai n K-(Ao) = 
{(0, 0), (—3, 0), (—3, 3), (—6, 0), (—6, 3)}. This procedure should be repeated for the 
remaining points of Ea^ leading to a total of 45 distinct edges. These 45 edges com- 
bined with the 45 central lattice points within (0) form a bipartite matching problem 
where the cost of assigning an edge to a central lattice point is given by the Euclidean 
distance between the mid point (or weighted mid point) of the edge and the central 
lattice point. 

Notice that for large index values, the sublattice point Aq E Aq is paired with 
points of Ai which are evenly distributed within a region Vt^ centered at Aq. If the 
product lattice A^r is based on the hypercubic lattice then forms a hypercube. In 
Chapters|5]and|6]we show that it is possible to change the design so that the sublattice 
points of Ai which are paired with a given Aq € Aq are evenly distributed within 
an L-dimensional hypersphere regardless of the choice of product lattice A^. The 
purpose of having the points spherically distributed is twofold; first, the side distortion 
is reduced and second, it allows a simple extension to more than two descriptions. 

Non Index-Assignment Based Designs 

To avoid the difficulty of designing efficient index-assignment maps it was suggested 
in [39] that the index assignments of a two-description system can be replaced by 
successive quantization and linear estimation. More specifically, the two side descrip- 
tions can be linearly combined and further enhanced by a refinement layer to yield 
the central reconstruction. The design of [39] suffers from a rate loss of 0.5 bit/dim. 
at high resolution and is therefore not able to achieve the MD rate-distortion bound. 
Recently, however, this gap was closed by Chen et al. [17, 18] who showed that by 
use of successive quantization and source splittin^^ it is indeed possible to achieve 
the two-channel MD rate-distortion bound, at any resolution, without the use of index 
assignments. Chen et al. recognized that the rate region of the MD problem forms 
a polymatroid and showed that corner points of this rate region can be achieved by 

Source splitting denotes tlie process of splitting a source X into two or more source variables, e.g. 
X (Xi , X2 ) where X ^ Xi X2 forms a Markov chain (in that order) [18]. 
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Figure 4.16: A central lattice A {dots), a sublattice Ao (squares) of index 5, a sublattice (cir- 
cles) of index 9, and a product lattice (stars) of index 45. The solid lines denote the Voronoi cell 

14(0) of the product lattice point located at the origin. Notice that 14(0) contains 45 central 
lattice points. The dashed lines denotes 14(0) shifted so it is centered at (—3, 1). 



successive estimation and quantization. This design is inherentiy asymmetiic in the 
description rate since any comer point of a non-trivial rate region will lead to asym- 
metric rates. It is therefore necessary to perform source splitting in order to achieve 
symmetry in the description rate. When finite-dimensional quantizers are employed 
there is a space-filHng loss due to the fact that the quantizer's Voronoi cells are not 
completely spherical and each description therefore suffers a rate loss. The rate loss 
of the design given in [17,18] is that of 2K — 1 quantizers because source splitting is 
performed by using an additional K — 1 quantizers besides the conventional K side 
quantizers. In comparison, the designs of the two-channel schemes based on index 
assignments [27,28, 120, 139] suffer from a rate loss of only that of two quantizers 
and furthermore, in the symmetric case, they suffer from a rate loss of only that of 
two spherical quantizers. That it indeed is possible to avoid source splitting in the 
symmetric case without the use of index assignments was recently shown by 0ster- 
gaard and Zamir [127] who constructed a if-channel symmetric MD scheme based on 
dithered Delta-Sigma quantization. The design of [127] is able to achieve the entire 
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symmetric two-channel MD rate-distortion region at any resolution and the rate loss 
when finite-dimensional quantizers are used is that of two lattice quantizers. Hence, in 
the two-channel case the rate loss when using index assignments is less than or equal 
to that of the designs which are not using index assignments [17, 18, 127]. 



Chapter 



/C-Channel Symmetric Lattice 
Vector Quantization 

In this chapter we consider a special case of the general if-channel symmetric MD 
problem where only a single parameter controls the redundancy tradeoffs between the 
central and the side distortions. With a single controlling parameter it is possible to 
describe the entire symmetric rate-distortion region for two descriptions and at high 
resolution, as shown in [120, 139], but it is not enough to describe the symmetric 
achievable A'-channel rate-distortion region. As such the proposed scheme offers a 
partial solution to the problem of designing balanced MD-LVQ systems. In Chapter]?] 
we include more controlling parameters in the design and show that the three-channel 
MD region given by Theorem l4. 1 . ll can be reached at high resolution. 

We derive analytical expressions for the central and side quantizers which, under 
high-resolution assumptions, minimize the expected distortion at the receiving side 
subject to entropy constraints on the side descriptions for given packet-loss proba- 
bilities. The central and side quantizers we use are lattice vector quantizers. The 
central distortion depends upon the lattice type in question whereas the side distor- 
tions only depend on the scaling of the lattices but are independent of the specific 
types of lattices. In the case of three descriptions we show that the side distortions can 
be expressed through the dimensionless normalized second moment of a sphere as 
was the case for the two descriptions system presented in [120, 139]. Furthermore, we 
conjecture that this is true in the general case of an arbitrary number of descriptions. 

In the presented approach the expected distortion observed at the receiving side 
depends only upon the number of received descriptions, hence the descriptions are 
mutually refinable and reception of any k out of K descriptions yields equivalent 
expected distortion. This is different from successive refinement schemes [32] where 
the individual descriptions often must be received in a prescribed order to be able to 
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refine each other, i.e. description number I will not do any good unless descriptions 
0, ... ,1—1 have already been received. We construct a scheme which for given packet- 
loss probabilities and a maximum bit budget (target entropy) determines the optimal 
number of descriptions and specifies the corresponding quantizers that minimize the 
expected distortion. 

5.1 Preliminaries 

We consider a central quantizer and K > 2 side quantizers. The central quantizer is 
(based on) a lattice Ac C with a fundamental region of volume i' — det(Ac). The 
side quantizers are based on a geometrically-similar and clean sublattice A^. C Ac of 
index N = |Ac/As| and fundamental regions of volume i^s = vN . The trivial case 
K — 1 leads to a single-description system, where we would simply use one central 
quantizer and no side quantizers. 

We will consider the balanced situation, where the entropy R is the same for 
each description. Furthermore, we consider the case where the contribution Di,i — 
0, ■ - -J K — 1 of each description to the total distortion is the same. Our design makes 
surg^l that the distortion observed at the receiving side depends only on the number of 
descriptions received; hence reception of any k out of K descriptions yields equivalent 
expected distortion. 

5.1.1 Index Assignments 

A source vector x is quantized to a reconstruction point Ac in the central lattice Ac. 
Hereafter follows index assignments (mappings), which uniquely map Ac to one vector 
(reconstruction point) in each of the side quantizers. This mapping is done through 
a labeling function a, and we denote the individual component functions of a by ai, 
where i — 0, . . . , K — 1. In other words, the injective map a that maps Ac into 
Ag X • • • X As, is given by 

a(Ac) = (ao(Ac),ai(Ac), . . .,aK~i{Xc)) (5.1) 
= (Ao,Ai,...,Ak-i), (5.2) 

where ai(Ac) = Ai G As and i = Q, . . . , K— 1. Each iiT-tuple (Aq, . . . , \k-i) is used 
only once when labeling points in Ac in order to make sure that Ac can be recovered 
unambiguously when all K descriptions are received. At this point we also define the 
inverse component map, a^^^, which gives the set of distinct central lattice points a 
specific sublattice point is mapped to. This is given by 

ar^(^0 = {Ac e Ac : a,(Ac) = AJ, A, £ A^, (5.3) 

^*We prove this symmetry property for the asymptotic case of ^ oo and Us 0. For finite A'^ we 
cannot guarantee the existence of an exact symmetric solution. However, by use of time-sharing arguments, 
it is always possible to achieve symmetry. 
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where |a-^(A,)| « N, since there are N times as many central lattice points as 
sublattice points within a bounded region of . 

Since lattices are infinite arrays of points, we construct a shift invariant labeling 
function, so we only need to label a finite number of points as is done in [28, 139]. 
Following the approach outlined in Chapter|2]we construct a product lattice which 
has N"^ central lattice points and N sublattice points in each of its Voronoi cells. 
The Voronoi cells V^r of the product lattice A^r are all similar so by concentrating on 
labeling only central lattice points within one Voronoi cell of the product lattice, the 
rest of the central lattice points may be labeled simply by translating this Voronoi cell 
throughout M^. Other choices of product lattices are possible, but this choice has 
a particular simple construction. With this choice of product lattice, we only label 
central lattice points within F7r(0), which is the Voronoi cell of A^r around the origin. 
With this we get 

a{Xc + X„) = a(Ac) + A^, (5.4) 
for all Att e Att and all Ac G Ac. 



5.2 Rate and Distortion Results 

Central Distortion 

Let us consider a scalar process that generates i.i.d. random variables with probability 
density function (pdf) /. Let X £ be a random vector made by blocking outputs 
of the scalar process into vectors of length L, and let x G denote a realization of 
X. The L-fold pdf of X is denoted fx and given b\P^ 

L-l 

The expected distortion (per dimension) Dc occuring when all packets are received is 
called the central distortion and is defined as 

^--T J2 I \\x-M?fx{x)dx, (5.6) 

where Vc{\c) is the Voronoi cell of a single reconstruction point Ac G Ac. Using 
standard high resolution assumptions, cf. Chapter[3] we may assume that each Voronoi 
cell is sufficiently small and fx {x) is smooth and hence approximately constant within 
each cell. In this case Ac is approximately the centroid (conditional mean) of the 
corresponding cell, that is 

/y (A )Xfxix)dx 

''it is worth pointing out that we actually only require the individual vectors to be i.i.d. and as such 
correlation within vectors is allowed. 



68 



(Chapters) 



iiT-Channel Symmetric Lattice Vector Quantization 



Since the pdf is approximately constant within a small region we also have that 

fxix)^ fxiXc), Vxey,(Ae), (5.8) 
and we can therefore express the probability, P, of a cell as 



P{V,{K))= fx{x)dx^ fx{K) dx = vfx{K). (5.9) 

where v is the volume of a Voronoi cell. With this, we get 

,5.10) 

V 

Inserting (|5.10l l into ( 15.61 ) gives 

^e«7 E ^(^c(A.)) / ^^^dx, (5.11) 

where Ac is a lattice so all Voronoi cells are congruent and the integral is similar for 
all Ac's. Hence, without loss of generality, we let Ac = and simplify (15.111 1 as 



n..U (5.12) 



L 



where we used the fact that P(Vc(Ac)) = 1. We can express the average 

central distortion (15.12b in terms of the dimensionless normalized second moment of 
inertia G(Ac) by 

D.^GiKV^. (5.13) 

Side Distortions 

The side distortion for the i^^ description, i.e. the distortion when reconstructing using 
only the description, is given by [139] 

A = Y V / \\x-a,{K)ffx{,x)dx, z = 0,...,if-l, 



7 / \\x - \c + Xc- cti{\c)\\'^ fx{x)dx 



7 E / \\x-\cffx{x)dx + ]- V / \\K-a,{K)ffx{x)dx 
+ 7" E / (a; - Ac, Ac - ai(Ac))/x(a;)da: 
^- + 7 E l|Ac-a^(Ac)|pP(Ac) 



L 

AcSAc 
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+ T y2 \ f xfx{x)dx- [ Xcfx{x)dx,Xc - ai{Xc)) 
= ^- + i l|Ac-a.(Ae)fP(Ae), (5.14) 



AcGAc 



where P{Xc) is the probability that X will be mapped to Ac, i.e. Q{X) — Ac, and the 
last equality follows since by use of ( |5.7| l we have that 



xjx{x)dx~ I \cJx{x)dx^Q. (5.15) 

We notice from ( 15.141 1 that independent of which labeling function we use, the distor- 
tion introduced by the central quantizer is orthogonal (under high-resolution assump- 
tions) to the distortion introduced by the side quantizers. 

Exploiting the shift-invariance property of the labeling function (|5.4| i makes it 
possible to simplify ( |5.14| l as 

+ z E ^ E iiAc-«.(A.)f 

, , (5.16) 

^D^ + JPL E l|Ac-a«(Ac)|P, i = Q,...,K-l, 
A<:ev„(o) 

where we assume the region 14- (0) is sufficiently small so P(Xc) ~ P{Xtt)/N'^, for 
Ac € V7r(A7r). Notice that we assume P{\tt) to be constant only within each region 
V^(A,r), hence it may take on different values for each Att G A^^. 



Central Rate 

Let i?c = H{Q{X)) / L denote the minimum entropy (per dimension) needed for a 
single-description system to achieve an expected distortion of Dc, the central distor- 
tion of the multiple-description system as given by ( 15.131 1. 

The single-description rate i?c is given by 



i?c = -y V / fx{x)dx\ogJ f fxix)dx). (5.17) 
^A.eA.^^c(A.) V-^^^(^=) / 



Using that each quantizer cell has identical volume and assuming that fx (x) is 
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approximately constant within Voronoi cells of the central lattice Ac, it follows that 
-^c ~ -y / fx{x)dx log2 (/x(Ac)z^) 

= -7 E / fx{x)dx log^ if x{\c)) 

- 7 E / Mx)dx log2(i/) (5.18) 

= -y E / Ix{x)dx log2 (/x(Ac)) - Y log2(^) 



AceA,-^^c(Ac) 
MX)-il0g2M. 



Side Rates 



Let Ri = H{ai{Q{X))) / L denote the entropy (per dimension) of the i description, 
where i = . . . ,K —1. Notice that in the symmetric situation we have Rg = Ri,i £ 
{0,...,K-1}. 

The side descriptions are based on a coarser lattice obtained by scaling (and possi- 
bly rotating) the Voronoi cells of the central lattice by a factor of N. Assuming the pdf 
of X is roughly constant within a sublattice cell, the entropy of the i*'^ side description 
is given by 

i?, = -i E I E / fx{x)dxiogJ E / /^(^)'^^ 
= -7 El E / fx{x)dx log^ (ufxixm 

= -7E E / fx{x)dx log^ if x{Xi)) 



7 E f E / fx{x)dxlog,iuN)] 



(5.19) 



The entropy of the side descriptions is related to the entropy of the single-description 
system by 

Ri = Rc-^log^{N). (5.20) 
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The index assignment is done by a labeling function a, that maps central lattice points 
to sublattice points. An optimal index assignment minimizes a cost functional when 
< K < K descriptions are received. In addition, the index assignment should be in- 
vertible so the central quantizer can be used when all descriptions are received. Before 
defining the labeling function we have to define the cost functional to be minimized. 
To do so, we first describe how to approximate the source sequence when receiving 
only K descriptions and how to determine the expected distortion in that case. Then 
we define the cost functional to be minimized by the labeling function a and describe 
how to minimize it. 

5.3.1 Expected Distortion 

At the receiving side, X E is reconstructed to a quality that is determined only 
by the number of received descriptions. If no descriptions are received we reconstruct 
using the expected value, EX, and if all K descriptions are received we reconstruct 
using the inverse map (Aq, . . . , Xk-i), hence obtaining the quality of the central 
quantizer. 

In this work we use a simple reconstruction rule which applies for arbitrary 
sourcesl^ When receiving 1 < k < K descriptions we reconstruct using the av- 
erage of the K descriptions. We show later (Theorem 15.3. Il l that using the average 
of received descriptions as reconstruction rule makes it possible to split the distortion 
due to reception of any number of descriptions into a sum of squared norms between 
pairs of lattice points. Moreover, this lead to the fact that the side quantizers' per- 
formances approach those of quantizers having spherical Voronoi cells. There are in 
general several ways of receiving k out of K descriptions. Let ^(^•'*) denote an 
index set consisting of all possible k combinations out of {0, . . . , K — 1}. Hence 



^ ^if)^ jjgjjotg jjjj element of .^(■^•'^) by I ^ {Iq, . . . Jn-i} e 



Our objective is to minimize some cost functional subject to entropy constraints 
on the description rates. We can, for example, choose to minimize the distortion 
when receiving any two out of three descriptions. Another choice is to minimize 
the weighted distortion over all possible description losses. In the following we will 
assume that the cost functional to be minimized is the expected weighted distortion 

'*We show in Chapter|2]that this simple reconstruction rule is, at high resolution, optimal in the quadratic 
Gaussian case, i.e. we show that the largest known three-channel MD region can be achieved in that case. 
This is in line with Ozarow's double-branch test-channel, where the optimum post filters are trivial at high 
resolution. 



Upon reception of any k descriptions we reconstruct X using 




(5.21) 



i=o 
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over all description losses and we further assume that the weights are given by the 
packet-loss probabilities. We discuss the case where the weights are allowed to be 
chosen almost arbitrarily in Chapter|6] 

Assuming the packet-loss probabilities, say p, are independent and are the same for 
all descriptions, we may use (15.161 1 and write the expected distortion when receiving 
K out K descriptions as 



Dc 



1 1 



E E 



K— 1 

Ac--E^'. 

K ^ ^ 



3=0 



(5.22) 



where \i. = ai - (Ac) and the two cases n e {0, if}, which do not involve the index- 
assignment map, are given by 1)1^'°^ « p^i?||X|p/L and dI^'^'' « {1 — p)^ Dc. 



5.3.2 Cost Functional 



From ( 15.221 ) we see that the distortion expression may be split into two terms, one 
describing the distortion occurring when the central quantizer is used on the source, 
and one that describes the distortion due to the index assignment. An optimal index 
assignment jointly minimizes the second term in ( 15.22b over all 1 < k < if — 1 pos- 
sible descriptions. The cost functional J*^^^ to be minimized by the index assignment 
algorithm is then given by 

K-l 

(5.23) 



where 



[l^pYpK- 



E 



E 



3=0 



(5.24) 



The cost functional should be minimized subject to an entropy constraint on the side 
descriptions. We remark here that the side entropies depend solely on ly and N and as 
such not on the particular choice of if-tuples. In other words, for fixed N and v the 
index assignment problem is solved if ( |5.23t is minimized. The problem of choosing 
ly and N such that the entropy constraint is satisfied is independent of the assignment 
problem and deferred to Section l5.4.2l 

The following theorem makes it possible to rewrite the cost functional in a way 
that brings more insight into which /sT-tuples to useP^ 



^'Notice that Theorem l5.3.1l is very general. We do not even require Ac or As to be lattices, in fact, they 
can be arbitrary sets of points. 



Section 5.3 Construction of Labeling Function 



73 



Theorem 5.3.1. For 1 < n < K we have 



E E 




Proof. See Appendix lH.il 

From Theorem 15. 3. II it is clear that ( |5.24| l can be written as 



□ 



L7V2 




E 

Acev,(o) 



i=0 



+ E 



AcGy„(0) 



K -K 



K^k{K - 



K-2 K-l 



(5.25) 



ly E E 



i=0 j=i+l 



The first term in (15.251 1 describes the distance from a central lattice point to the 
centroid of its associated JC-tuple. The second term describes the sum of pairwise 
squared distances (SPSD) between elements of the iiT-tuples. In Section 15.41 (by 
Proposition |5.4.2| i we show that, under a high-resolution assumption, the second term 
in ( 15.251 1 is dominant, from which we conclude that in order to minimize (15.231 1 we 
have to choose the i^T-tuples with the lowest SPSD. These A'-tuples are then assigned 
to central lattice points in such a way, that the first term in ( 15.251 1 is minimized. 

Independent of the packet-loss probability, we always minimize the second term 
in ( 15.251 1 by using those JC-tuples that have the smallest SPSD. This means that, at 
high resolution, the optimal iiT-tuples are independent of packet-loss probabilities and, 
consequently, the optimal assignment is independenj^ of the packet-loss probability. 



5.3.3 Minimizing Cost Functional 

In order to make sure that a is shift-invariant, a given _fC-tuple of sublattice reconstruc- 
tion points is assigned to only one central lattice point Ac G Ac. Notice that two 
ii'-tuples which are translates of each other by some A^r € A^ must not both be 
assigned to central lattice points located within the same region V.,^{\-,t), since this 
causes assignment of the same iC-tuples to multiple central lattice points. The region 
V^(0) will be translated throughout and centered at A^r G A^r, so there will be 
no overlap between neighboring regions, i.e. ( A^ ) n ( A^' ) = 0, for A^ , A^' G A„ 
and A^ 7^ A^'. One obvious way of avoiding assigning iiT-tuples to multiple central 

Given the central lattice and the sublattice, the optimal assignment is independent of p. However, we 
show later that the optimal sublattice index N depends on p. 
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lattice points is then to exclusively use sublattice points located within 14^(0). How- 
ever, sublattice points located close to but outside K-(O), might be better candidates 
than sublattice points within T/n-(0) when labeling central lattice points close to the 
boundary. A consistent way of constructing iiT-tuples, is to center a region V at all 
sublattice points Aq G n V,r{Q), and construct AT-tuples by combining sublattice 
points Xi E As , i — I , ■ ■ ■ , K ~ 1 within V^(Ao) in all possible ways and select the 
ones that minimize (15.251 1. This is illustrated in Fig. 15.11 For a fixed G As, the 
expression X^a eA nv{\ ) W^i ^ ^jll^ minimized when V forms a sphere centered 
at Xi. Our construction allows for V to have an arbitrary shape, e.g. the shape of V,r 
which is the shape used for the two-description system presented in [28]. However, if 

V is not chosen to be a sphere, the SPSD is in general not minimized. 

For each Ao G As n 14- (0) it is possible to construct N-^^^ X-tuples, where N 
is the number of sublattice points within the region V. This gives a total of NN^^^ 
if-tuples when all Ag G As n K^(O) are used. However, only iV^ central lattice points 
need to be labeled (14- (0) only contains N'^ central lattice points). When K = 2,we 
let iV = TV, so the number of possible if -tuples is equal to N^, which is exactly the 
number of central lattice points in V^ttIO). In general, for K > 2, the volume i) of 

V is smaller than the volume of V^iO) and as such N < N. We can approximate 
N through the volumes and D, i.e. N « i^/j^s- To justify this approximation let 
A C be a real lattice and let v = det(A) be the volume of a fundamental region. 
Let S{c, r) be a sphere in of radius r and center c G R^. According to Gauss' 
counting principle, the number Az of integer lattice points in a convex body in 
equals the volume Vo^*^) of with a small error term [92]. In fact if = S{c, r) 
then by use of a theorem due to Minkowski it can be shown that, for any c G 
and asymptotically as r ^ cxd, Az{r) = Vol(S'(c, r)) — Lo^r^, where cj^ is the 
volume of the L-dimensional unit sphere [40], see also [11,33,62,77, 144]. It is 
also known that the number of lattice points Aa (n) in the first n shells of the lattice 
A satisfies, asymptotically as n ^ cx), Af^{n) = lliiji^I'^ jv [139]. Hence, based 
on the above we approximate the number of lattice points in y by vjvs, which is an 
approximation that becomes exact as the number of shells n within V goes to infinit\l^ 
(which corresponds to ^ oo). Our analysis is therefore only exact in the limiting 
case of Af ^ oo. With this we can lower bound v by 



Hence, V contains N > N^^ sublattice points so that the total number of possi- 
ble isT-tuples is NN^-^ > N^. 

In Fig. 15. H is shown an example of V and V,r regions for the two-dimensional 
lattice. In the example we used K — 3 and A^ = 25, hence there are 25 sublattice 

''For the high-resolution analysis given in Section |T4l it is important that i> is kept small as the number 
of lattice points within V goes to infinity. This is easily done by proper scaHng of the lattices, i.e. making 
sure that Us — > as Af ^ oo. 



lim £> > iysN^^'^^~^\ 



(5.26) 



N^oo 
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points within V^r- There are N = iV^/'^ ^' = 5 sublattice points in V which is 
exactly the minimum number of points required, according to ( 15.26b . 




Figure 5.1: The region V (big circles) is here shown centered at two different sublattice points 
within Vtt{0). Small dots represents sublattice points of As and large dots represents product 
lattice points A,r G A,r. Central lattice points are not shown here. V-^ (shown as squares) 
contains 25 sublattice points centered at product lattice points. In this example V contains 5 
sublattice points. 

With equahty in ( 15.26b we obtain a region that contains the exact number of sub- 
lattice points required to construct tuples for each of the N Xq points in VT^iO). 
According to (15.251 ). a central lattice point should be assigned that iC-tuple where a 
weighted average of any subset of the elements of the X-tuple is as close as possible to 
the central lattice point. The optimal assignment of X-tuples to central lattice points 
can be formulated and solved as a linear assignment problem [151]. 



Shift-Invariance by use of Cosets 

By centering V around each Aq G n V^(0), we make sure that the map a is 
shift-invariant. However, this also means that all /-C-tuples have their first coordinate 
(i.e. Ao) inside 14- (0). To be optimal this restriction must be removed which is easily 
done by considering all cosets of each X-tuple. The coset of a fixed A'-tuple, say 
t = (Ao,Ai,...,Ak_i) where Ao G A,, n V;,(0) and (Ai, . . . , A^-i) G Af-^, 
is given by Coset(i) — + A^ : VA^r G A^} . AT-tuples in a coset are distinct 
modulo Att and by making sure that only one member from each coset is used, the 
shift-invariance property is preserved. In general it is sufficient to consider only those 
Att product lattice points that are close to V^(0), e.g. those points whose Voronoi cell 
touches VTr{Q). The number of such points is given by the kissing-number .S(A7r) of 
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the particular lattice [22]. 



Dimensionless Expansion Factor iJjl 

Centering V around Ao points causes a certain asymmetry in the pairwise distances of 
the elements within a A'-tuple. Since the region is centered around Aq the maximum 
pairwise distances between Aq and any other sublattice point will always be smaller 
than the maximum pairwise distance between any two sublattice points not including 
Aq. This can be seen more clearly in Fig. 15. 21 Notice that the distance between the pair 
of points labeled (Ai, A2) is twice the distance of that of the pair (Aq, Ai) or (Aq, A2). 
However, by slightly increasing the region V to also include A2 other tuples may be 
made, which actually have a lower pairwise distance than the pair (Ai, A2). For this 
particular example, it is easy to see that the 3-tuple t = (Aq, Ai, A2) has a greater 
SPSD than the 3-tuple t' = (Ao, Ai, A^). 




' V 



Figure 5.2: The region V is here centered at the point Aq. Notice that the distance between Ai 
and A2 is about twice the maximum distance from Aq to any point in D V . The dashed circle 
illustrates an enlargement of V . 

For each Ao G Vt^{Q) we center a region V around the point, and choose those 
N i^-tuples, that give the smallest SPSD. By expanding V new if-tuples can be 
constructed that might have a lower SPSD than the SPSD of the original N i^T-tuples. 
However, the distance from Ao to the points farthest away increases as V increases. 
Since we only need N ii'-tuples, it can be seen that V should never be larger than twice 
the lower bound in ( 15.261 1 because then the distance from the center to the boundary of 
the enlarged V region is greater than the maximum distance between any two points 
in the V region that reaches the lower bound. In order to theoretically describe the 
performance of the quantizers, we introduce a dimensionless expansion factor 1 < 
V'L < 2 which describes how much V must be expanded from the theoretical lower 
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bound ( 15.261 1, to make sure that N optimal iC-tuples can be constructed by combining 
sublattice points within a region V. 

For the case of K = 2 we always have i/jl = 1 independent of the dimension L 
so it is only in the case K > 3 that we need to find expressions for ipL- 

Theorem 5.3.2. For the case of if = 3 and any odd L, the dimensionless expansion 
factor is given by 

/ X 1/2L / r , 1 \ 1/2L 

^L=(^) (^) (5.27) 



\uJL-i J \ 2L 
where is the volume of an L-dimensional unit sphere and (3l is given by 

L + \ L-1 



n=0 ^ ^ fe=0 \ 2 )k 

X 



(5.28) 



Proof. See AppendixiRll □ 

For the interesting case of L ^ cx3 we have the following theorem. 

Theorem 5.3.3. For K = "i and L ^ oo the dimensionless expansion factor '0l is 
given by 



V'oo = j . (5.29) 

Proof. See AppendixlHj] □ 
Table l5.l llistJ^T/'r for K ~ i and different values of L and it may be noticed that 

Remark 5.3.1. In order to extend these results to if > 3 it follows from the proof 
of Theorem 15.3.21 that we need closed-form expressions for the volumes of all the 
different convex regions that can be obtained by if — 1 overlapping spheres. With 
such expressions it should be straightforward to find V'l for any K. However, the 
analysis of t/jl for the case of if = 3 (as given in the proof of Theorem 15.3.21 ) is 
constructive in the sense that it reveals how can be numerically estimated for any 
if and L, see Appendix|F] 

Remark 5.3.2. In order to achieve the shift-invariance property of the index- 
assignment algorithm, we impose a restriction upon Ao points. Specifically, we require 
that Aq e Vt^{Q) so that the first coordinate of any if -tuple is within the region ^-(0). 



'^''Theorem l5.3.2l is only valid for L odd. However, in the proof of Theorem l5.3.2l it is straightforward to 
replace the volume of spherical caps by standard expressions for circle cuts in order to obtain 1I12 ■ 
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L 




L 




1 


1.1547005--- 


15 


1.1036412- • • 


2 


1.1480804- • • 


17 


1.1016878- •• 


3 


1.1346009- • • 


19 


1.1000271 • • • 


5 


1.1240543- • • 


21 


1.0985938 • • • 


7 


1.1172933- • • 


51 


1.0883640- •• 


9 


1.1124896- • • 


71 


1.0855988 • • • 


11 


1.1088540- • • 


101 


1.0831849- •• 


13 


1.1059819- •• 


oo 


1.0745699... 



Table 5.1: tpL values obtained by use of Theorems 15. 3.2\ and \5. 3.3\f or it = 3. 

To avoid excluding if-tuples that have their first coordinate outside (0) we form 
cosets of each iiT-tuple and allow only one member from each coset to be assigned to 
a central lattice point within V^(0). This restriction, which is only put on Aq G A^, 
might cause a bias towards Ao points. However, it is easy to show that, asymptotically 
as ^ oo, any such bias can be removed. For the case of i^T = 2 we can use sim- 
ilar arguments as used in [28], and for iiT > 2, as shown in Chapter|6] the number 
of if-tuples affected by this restriction is small compared to the number of iiT-tuples 
not affected. So for example this means that we can enforce similar restrictions on all 
sublattice points, which, asymptotically as TV — > oo, will only reduce the number of 
if-tuples by a neglectable amount. And as such, any possible bias towards the set of 
points Ao € As is removed. 

As mentioned above, the AT-tuples need to be assigned to central lattice points 
within 14^(0). This is a standard linear assignment problem where a cost measure 
is minimized. However, solutions to linear assignment problems are generally not 
unique. Therefore, there might exist several labelings, which all yield the same cost, 
but exhibit a different amount of asymmetry. Theoretically, exact symmetry may then 
be obtained by e.g. time sharing through a suitable mixing of labelings. In practice, 
however, any scheme would use a finite A^ (and finite rates). In addition, for many 
applications, time sharing is inconvenient. In these non-asymptotic cases we cannot 
guarantee exact symmetry. To this end, we have provided a few examples that as- 
sess the distortions obtained from practical experiments, see Section 15.61 (Tables 15.31 
and|54]l. 

5.4 High-Resolution Analysis 

In this section we derive high-resolution approximations for the expected distortion. 
For this high-resolution analysis we let ^ oo and Vg 0. Thus, the index N of 
the sublattices increases, but the actual volumes of the Voronoi cells shrink. 
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5.4.1 Total Expected Distortion 

We wish to obtain an analytical expression for the expected distortion given by ( 15.221 1. 
In order to achieve this we first relate the sum of distances between pairs of sublattice 
points to G{Sl), the dimensionless normalized second moment of an i-sphere. This 
is done by Proposition |5AT] 

Proposition 5.4.1. For K — 2 and asymptotically as ^ oo and i^s 0, as well 

as for K = 3 and asymptotically as A^, i ^ cxd and i^s 0, we have for any pair 

(A„A,), i,j = 0,...,K-l, i^j. 

Proof. See Appendix [H.4l □ 

Conjecture 5.4.1. Proposition |5.4. l| is true also for K > 3 asymptotically as N,L ^ 
oo and 0. 



Remark 5.4. 1. Arguments supporting coniecture l5.4. H are given in Appendix lH.4l 
Remark 5.4.2. In Appendix IH. 41 we also present an exact expression for Proposition 
l5AT] for = 3 and finite L. 

Recall that we previously showed that by use of Theorem 15.3.11 it is possible to 
split ( 15.221 1 into two terms; one that describes the distance from a central lattice point to 
the centroid of its associated iiT-tuple and another which describes the sum of pairwise 
squared distances (SPSD) between elements of the if -tuples. To determine which of 
the two terms that are dominating we present the following proposition: 

Proposition 5.4.2. For N ^ oo and 2 < ii' < oo we have 

2 



AoGy„(0) 



A. 



1 V-K-i , 



0. 



Proof. See Appendix |H. 51 

The expected distortion ( 15.221 1 can by use of Theorem l5.3.1l be written as 



(5.30) 

□ 




(5.31) 
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By use of Proposition l5.4. IK as an approximation that becomes exact for L oo), 
Proposition l5.4.2l and Eq. ( 15.13b it follows that (15.31b can be written as 



A^'^I^ E (lf.|7f^)'i: EllA.-A.II^] (532, 



X (^G(A,)^2/L ^ {^-^ G(5i)Vi^^^/^(^-^)i^^/^) . (5.33) 
The second term in (15.331 ). that is 

G(5i)Vi7V2^/^(^-iV2/^ (5.34) 

is the dominating term for k < K and N ^ oo and describes the side distortion due 
to reception of any k < K descriptions. Observe that this term is only dependent upon 
K through the coefficient -fc^ . 

The total expected distortion oi^^ is obtained from (15.33b by summing over k 
including the cases where k = and k ^ K, which leads to 

« KiGiA.y^'^ + 7i:2G(5i)ViiV'^/^(^-^^i^'/^ + p^'EWXWyL, (5.35) 
where Ki is given by 



K=l ^ 

K 



(5.36) 



and K2 is given by 

K — 1 ^ ^ 

Using ( 15.18b and ( 15. 19b we can write v and iV as a function of differential entropy 
and side entropies, that is 

^22iMx)^R.)^ (5.38) 

and 
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where Rg ~ Ri,i = 0, . . . , K — 1 denotes the side description rate. Inserting (15.381 1 
and ( |5.39t in ( 15.35b makes it possible to write the expected distortion as a function of 
entropies 

7^1^) « i^iG(A,)22(''(^)-«=) 

(5.40) 

where we see that the distortion due to the side quantizers depends only upon the 
scaling (and dimension) of the sublattice but not upon which sublattice is used. Thus, 
the side distortions can be expressed through the dimensionless normalized second 
moment of a sphere. 



5.4.2 Optimal u, N and K 

We now derive expressions for the optimal i^, N and K. Using these values we are 
able to construct the lattices Ac and Ag. The optimal index assignment is hereafter 
found by using the approach outlined in Section 15.31 These lattices combined with 
their index assignment completely specify an optimal entropy-constrained MD-LVQ 
system. 

In order for the entropies of the side descriptions to be equal to the target entropy 
Rt I K, we rewrite ( |5.19t and get 

Nv = 2^(''W~«-/^) ^ r, (5.41) 

where t is constant. The expected distortion ( 15 .40b may now be expressed as a 
function of v, 

D(J<^ = XiG(A,);.2/L 

+ k2i^lG{SL)iy'^^^iy^^^^T^TT^^^ + p^E\\X\\^/L. 

Differentiating w.rt. v and equating to zero gives, 

2 - , .V^l^ /2 2 if \ - o , 2K 2g 

= -i^iG A, + - - -— — K2^jIG{Sl) ^-TW^r^TK^, 

L V \L L K — 1 J V 

(5.43) 

from which we obtain the optimal value of v 

..-( 1 ,5.44, 
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The optimal N follows easily by use of (I5.41l i 

K2 G{Sl) Vl J 

Eq. (15.451) shows that the optimal redundancy N is, for fixed K, independent of the 
sublattice as well as the target entropy. 

For a fixed K the optimal v and TV are given by ( 15.441 1 and (15.45b . respectively, 
and the optimal K can then easily be found by evaluating ( 15.351) for various values of 
K, and choosing the one that yields the lowest expected distortion. The optimal K is 
then given by 

A'opt = arg rmn Df^ , K = l,..., K^,,, (5.46) 

where Ki^^y, is a suitable chosen positive integer. In practice K will always be finite 
and furthermore limited to a narrow range of integers, which makes the complexity of 
the minimization approach, given by ( 15.461 ). negligible. 

5.5 Construction of Practical Quantizers 
5.5.1 Index Values 

Eqs. ( 15.44b and ( 15.451 ) suggest that we are able to continuously trade off central versus 
side-distortions by adjusting N and v according to the packet-loss probability. This 
is, however, not the case, since certain constraints must be imposed on N . First of 
all, since denotes the number of central lattice points within each Voronoi cell of 
the sublattice, it must be integer and positive. Second, we require the sublattice to 
be geometrically similar to the central lattice. Finally, we require the sublattice to 
be a clean sublattice, so that no central lattice points are located on boundaries of 
Voronoi cells of the sublattice. This restricts the amount of admissible index values 
for a particular lattice to a discrete set, cf. Section l2.3.1l 

Fig. 153] shows the theoretically optimal index values (i.e. ignoring the fact that N 
belongs to a discrete set) for the A2 quantizer, given by ( 15.45b for ipL — 1, 1.1481 
and 1.1762 corresponding io K — 2,3 and 4, respectivelyCj Also shown are the 
theoretical optimal index values when restricted to admissible index values. Notice 
that the optimal index value N increases for increasing number of descriptions. This 
is to be expected since a higher index value leads to less redundancy; this redun- 
dancy reduction, however, is balanced out by the redundancy increase resulting from 
the added number of descriptions. In [103] we observed that for a two-description 
system, usually only very few index values would be used (assuming a certain min- 
imum packet-loss probability). Specifically, for the two-dimensional A2 quantizer, 

■*'The value 1/1 = 1.1762 for K = i is estimated numerically by using the method outlined in Ap- 
pendixlFl 
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■ - K= 2: Theoretical N 

— K= 3: Theoretical N 

— K= 4: Theoretical N 

— Admissible 
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Figure 5.3: Theoretical optimal index values for the A2 quantizer as a function of the 
packet-loss probability. The thin solid lines are obtained by restricting the theoretical op- 
timal index values given by | |5.45I ) to be (optimal) admissible values given by the set N — 
{1, 7, 13, 19, 31, 37, 43, 49, . . . }. The optimal admissible index values are those that mini- 
mize is. 3 51 for a given p. 



only N G {1,7, 13} was used, while for higher dimensional quantizers greater index 
values would be used. However, here we see that by increasing the number of de- 
scriptions beyond if = 2, it is optimal to use greater index values which adds more 
flexibility to the scheme. 

From Fig. 15.31 it can be seen that when the continuous optimal index value is 
rounded to the optimal admissible index value it is always the closest one from either 
below or above. This means that, at least for the A2 lattice, the optimal admissible in- 
dex value is found by considering only the two values closest to the continuous index 
value, and using the one that minimizes < I5.35I ). 

5.5.2 Constructing K-tuples 

The design procedure for constructing if -tuples as described in Section |533] can be 
summarized as follows: 

1. Center a sphere V at each Aq e n V^(0) and construct all possible if -tuples 
(Ao, . . . , Xk^i) where Xi S As, i — 1, . . . , K ~ 1. This makes sure that all 
if -tuples have their first coordinate (Aq) inside 14- (0) and they are therefore 
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shift invariant. We will only use AT-tuples whose elements satisfy || Ai — Aj || < 
r,yi, j G {0, . . . , K — 1}, where r is the radius of V. Make V large enough so 
that at least distinct if -tuples are found for each Aq. 

2. Construct cosets of each if-tuple. 

3. The central lattice points in Ac H (0) must now be matched to distinct K- 
tuples. This is a standard linear assignment problem where only one member 
from each coset is (allowed to be) matched to a central lattice point in T4- (0). 

The restriction ||Ai — Aj|| < r from step 1) which is used to avoid bias towards any 
of the sublattices, reduces the number of if-tuples that can be constructed within the 
sphere V. To be able to form N if-tuples it is therefore necessary to use a sphere V 
with a volume larger than the lower bound ( 15.26b . This enlargement is exactly given 
by tpL. As such, for each Aq, we form (at least) N K -tuples and these A'-tuples are the 
ones having minimum norm. We show later (see Lemma [6.2. 3 1 and its proof) that we 
actually form all such A'-tuples of minimal norm which implies that no other AT-tuples 
can improve the SPSD. 

5.5.3 Assigning K-Tuples to Central Lattice Points 

In order to assign the set of A'-tuples to the A^^ central lattice points we solve a linear 
assignment problem. However, for large A^, the problem becomes difficult to solve in 
practice. To solve a linear assignment problem or more specifically a bipartite match- 
ing problem, one can make use of the Hungarian method [78], which has complexity 
of cubic order. Hence, if the Hungarian method is used to solve the assignment prob- 
lem the complexity is of order 0{N^). We would like point out that letting A^^ = A^^ 
is a convenient choice, which is valid for any lattice. However, it is possible to let 
A^TT = N£^, where both A^ and ^ are admissible index values. In this case Nt^ is also 
guaranteed to be an admissible index value by Lemma 12.3.11 If ^ = 1 then N^^ ~ N, 
which is a special case where (0) contains a single sublattice point Aq of As @With 
Ntt = N^, the complexity is reduced to ff{N^). 

Vaishampayan et al. observed in [120, 139] that the number of central lattice points 
to be labeled can be reduced by exploiting symmetries in the lattices. For example, 
one can form the quotient ^-module A/A^ and only label representatives of the 
orbits of A/Att/F, where F is a group of automorphisms, cf. Chapter [21 While only 
two descriptions were considered in [139] it is straight-forward to show that their idea 
also works in our design for an arbitrary number of descriptions. This is because 

^^Practical experiments have shown that having too few sublattice points in Vt: (0) leads to a poor index 
assignment. Theoretically, we do not exclude the possibility that N^: = N, since we only require that V 
contains a large number of sublattice points but such a contraint is not imposed on Vir(O). However, in the 
following chapter, where we consider the asymmetric case (so there are several index values), the special 
case is not allowed. 
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we design the sublattices and product lattices as described in Chapter |2] hence the 
notion of quotient modules and group action are well defined. Since the order of the 
group r depends on the lattices but is independent of N, the complexity reduction 
by exploiting the symmetry of the quotient module is a constant multiplicative factor 
which disappears in the order notation. 

Alternatively, Huang and Wu [65] recently showed that for certain low dimen- 
sional lattices it is possible to avoid the linear assignment problem by applying a 
greedy algorithm, without sacrificing optimality. The complexity of the greedy ap- 
proach is on the order of ff{N), which is a substantial improvement for large N. 

In the present work we show that the assignment problem can always be posed and 
solved as a bipartite matching problem. This holds for any lattice in any dimension 
and it also holds in the asymmetric case to be discussed in Chapter|6] In a practical 
situation it might, however, be convenient to compute the assignments offline and 
tabulate for further use. 

5.5.4 Example of an Assignment 

In the following we show a simple assignment for the case of K = 2, N ~ 7 and the 
A2 lattice. Since iV = 7 we have N-^ — 49 and as such there is 49 central lattice points 
within V,r{0), see Fig. 15.41 The individual assignments are also shown in Table 15. 2] 

The assignments shown in Table 15. 2] are obtained by using the procedure outlined 
in Section 15.5.21 Since we have N^^ — 49 and K — 2 it follows that we have 7 
sublattice points of within T/,r(0) (one of them is the origin). Let us denote this set 
of sublattice points by E\_^ . 

1. Center a sphere V at the first element of E\_^, i.e. the origin. Pick the candidate 
sublattice points of As, i.e. those which are contained within V n As. We make 
sure that the radius of the sphere is so large that it contains Vtt{0)- Thus, we 
have at least as many sublattice points in V as in Vtt (0). Then form all possible 
distinct edges (2-tuples) having the origin (Aq) as first coordinate and Ai G 
K-('^o)n As as second coordinate. Notice that we have at least N edges. Repeat 
this for the remaining elements of E\_^ so that we end up having at least A^^ 
edges in total. 

2. Form the coset of each edge. Specifically, construct the following set of edges: 

Coset(Ao, Ai) = {(Ao + A^, Ai + A^) : A^ e A^}. (5.47) 

In practice we restrict each coset to contain a finite number of elements. In 
fact, we usually only require that the cardinality of the cosets is greater than 
^(As), the kissing number of the lattice. The product lattice points we use 
when constructing the cosets are then the R{As) + 1 points of smallest norm. 
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Figure 5.4: A central lattice Ac (dots) based on the A2 lattice and a sublattice (circles) of 
index 7. The hexagonal region (dashed lines) describes V,r(0). The solid lines connect pairs 
of sublattice points (also called 2-tuples or edges) and the dotted lines connect each edge to 
a central lattice point. A total of 49 edges (some overlaping each other) are shown and these 
edges are associated with the 49 central lattice points contained within (0). 

3. If we have more edges than central lattice points we introduce "dummy" cen- 
tral lattice points. In this way we have an equal amount of edges and central 
lattice points. The assignment of edges to central lattice points is now a straight 
forward bipartite problem, where the costs of the dummy nodes are set to zero, 
so that the optimal solutions are not affected. We note that only one element of 
each coset is used. In this way we preserve the shift invariance property of the 
assignments. We only keep the N^^ assignments belonging to true central lattice 
points and as such we discard the assignments (if any) that belong to "dummy" 
nodes. 

In Appendix |G] we show part of a complete assignment of a more complicated 
example. 

5.6 Numerical Results 

In this section we compare the numerical performances of two-dimensional entropy- 
constrained MD-LVQ (based on the A2 lattice) to their theoretical prescribed perfor- 
mances. 
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(-2,-1.73) 


(1.5, 0.87) 


(2, 1.73) 


(0, 0) 


(-1.5, -2.6) 


(-1.5,-4.33) 


(-2,-1.73) 


(2.5, 0.87) 


(2.5, -0.87) 


(2, 1.73) 


(-0.5, -2.6) 


(-2, -1.73) 


(0.5, -2.6) 


(-3,0) 


(-4.5, -0.87) 


(-2.5, 0.87) 


(0.5, -2.6) 


(0.5 - 2.6) 


(0.5, -2.6) 


(-2, 0) 


(-2,-1.73) 


(-2.5, 0.87) 


(1.5, -2.6) 


(3, -3.46) 


(0.5, -2.6) 


(-1,0) 


(-2.5, 0.87) 


(0, 0) 


(0, -3.46) 


(-1.5, -4.33) 


(0.5, -2.6) 








(1,-3.46) 


(1, -5.2) 


(0.5, -2.6) 



Table 5.2: A complete assignment for the 49 central lattice points contained within Vt, (0) for 
the case of K = 2 and N — 7. 



5.6.1 Performance of Individual Descriptions 

In the first experiment we design three-channel MD-LVQ based on the A2 quantizer 
We quantize an i.i.d. unit-variance Gaussian source which has been blocked into two- 
dimensional vectors. The number of vectors used in the experiment is 2 • 10^. The 
entropy of each side description is 5 bit/dim. and we vary the index value in the range 
31 - 67. The dimensionless expansion factor ipL is set to 1.14808, see Table ISTl 
The numerical and theoretical distortions when receiving only a single description 
out of three is shown in Table 15.31 Similarly, Table 15.41 shows the distortions of the 
same system due to reception of two out of three descriptions and Table 15.51 shows 
the performance of the central quantizer when all three descriptions are received. The 
column labeled "Avg." illustrates the average distortion of the three numerically mea- 
sured distortions and the column labeled "Theo." describes the theoretical distortions 
given by ( 15.341 ). It is clear from the tables that the system is symmetric; the achieved 



88 



(Chapters) 



iiT-Channel Symmetric Lattice Vector Quantization 



distortion depends on the number of received descriptions but is essentially indepen- 
dent of which descriptions are used for reconstruction. The numerically measured 
discrete entropies of the side descriptions are shown in Table 15. 6] 



N 


Ao 


Ai 


Aa 


Avg. 


Theo. 


31 


-25.6918 


-25.6875 


-25.6395 


-25.6729 


-24.8853 


37 


-24.5835 


-24.5324 


-24.5404 


-24.5521 


-24.5011 


43 


-24.5772 


-24.5972 


-24.5196 


-24.5647 


-24.1748 


49 


-24.2007 


-24.2837 


-24.2713 


-24.2519 


-23.8911 


61 


-23.8616 


-23.9011 


-23.8643 


-23.8757 


-23.4155 


67 


-23.7368 


-23.7362 


-23.7655 


-23.7462 


-23.2118 



Table 5.3: Distortion (in clB) due to reception of a single description out oftliree. 



N 


^Ao + Ai) 


|(Ao + A2) 


i(Al+A2) 


Avg. 


Theo. 


31 


-30.7792 


-30.7090 


-30.7123 


-30.7335 


-30.9059 


37 


-29.8648 


-29.8430 


-29.9472 


-29.8850 


-30.5217 


43 


-29.9087 


-29.8749 


-29.9641 


-29.9159 


-30.1954 


49 


-29.6290 


-29.5577 


-29.6662 


-29.6176 


-29.9117 


61 


-29.3076 


-29.2185 


-29.3715 


-29.2992 


-29.4361 


67 


-29.1752 


-29.2128 


-29.2151 


-29.2010 


-29.2324 



Table 5.4: Distortion (in dB) due to reception of two descriptions out of three. 



N 


Ac 


Theo. 


31 


-43.6509 


-43.6508 


37 


-44.4199 


-44.4192 


43 


-45.0705 


-45.0719 


49 


-45.6401 


-45.6391 


61 


-46.5879 


-46.5905 


67 


-46.9992 


-46.9979 



Table 5.5: Distortion (in dB) due to reception of all three descriptions. 

The distortions shown in Tables |53] to 15. SI correspond to the case where we vary 
the index value throughout the range 67 > iV > 31 for three-channel MD-LVQ 
operating at i?<; = 5 bit/dim. per description. To achieve similar performance with 
a (3,1) SCEC we need to vary the correlation pq within the interval —0.49 < Pq < 
—0.45, as shown in Fig. 15.51 
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N 


Ao 


Ai 


Aa 


31 


5.0011 


5.0012 


5.0012 


37 


4.9925 


4.9982 


4.9988 


43 


4.9967 


5.0006 


5.0006 


49 


4.9993 


5.0004 


5.0004 


61 


5.0018 


5.0017 


5.0017 


67 


5.0023 


5.0022 


5.0022 



Table 5.6: Numerically measured discrete entropies [bit/dim.] for the individual descriptions. 
Here the target description rate is set to 5 bit/dim. 



5.6.2 Distortion as a Function of Packet- Loss Probability 

We now show the expected distortion as a function of the packet-loss probability for 
isT-channel MD-LVQ where K = 1,2, 3. We block the i.i.d. unit-variance Gaussian 
source into 2 10^ two-dimensional vectors and let the total target entropy be 6 bit/dim. 
The expansion factor is set to ^^2 = 1 for K ~ 1,2 and ■02 ~ 1.14808 for K = 'd. 
We sweep the packet-loss probability p in the range p e [0; 1] in steps of 1/200 and 
for each p we measure the distortion for all admissible index values and use that index 
value which gives the lowest distortion. This gives rise to an operational lower hull 
(OLH) for each quantizer This is done for the theoretical curves as well by insert- 
ing admissible index values in ( 15.35b and use that index value that gives the lowest 
distortion. In other words we compare the numerical OLH with the theoretical OLH 
and not the "true'@ lower hull that would be obtained by using the unrestricted index 
values given by (15.451 1. The target entropy is evenly distributed over K descriptions. 
For example, for K — 2 each description uses 3 bit/dim., whereas for K ~ i each 
description uses only 2 bit/dim. The performance is shown in Fig. 15.61 The practical 
performance of the scheme is described by the lower hull of the if-curves. Notice 
that at higher packet-loss probabilities {p > 5%) it becomes advantageous to use three 
descriptions instead two. 

It is important to see that when the distortion measure is the expected distortion 
based on the packet-loss probability, then the notion of high resolution is slightly mis- 
leading. For example, if we let the rate go to infinity, then for a given fixed packet-loss 
probability p the only contributing factor to the expected distortion is the distortion 
due to the estimation of the source when all packets are lost. This term is given by 
so that for a unit- variance source, in the asymptotic case of i? ^ cx), 
the expected distortion is simply given by K10\ogiQ{p) dB. In other words, with a 
packet-loss probability of 10%, if the number of packets is increased by one, then 

'"A lattice is restricted to a set of admissible index values. This set is generally expanded when the lattice 
is used as a product quantizer, hence admissible index values closer to the optimal values given by j5.45) 
can in theory be obtained, cf. Section [2.3.1l 
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-40- 
-42- 
-44- 
-46- 

-0.49 -0.485 -0.48 -0.475 -0.47 -0.465 -0.46 -0.455 -0.45 

Figure 5.5: The simultaneously achievable one-channel, two-channel and three-channel dis- 
tortions for the unit-variance Gaussian source at 5 hit/dim. for —0.49 < pq < —0.4:5 for a 
(3, 1) SCEC. 



the corresponding decrease in distortion is exactly 10 dB. We have illustrated this in 
Fig.Oforii'^^ 1,...,5. 

5.7 Conclusion 

We derived closed-form expressions for the central and side quantizers which, at high- 
resolution conditions, minimize the expected distortion of a symmetric if-channel 
MD-LVQ scheme subject to entropy constraints on the side descriptions for given 
packet-loss probabilities. The expected distortion observed at the receiving side de- 
pends only upon the number of received descriptions but is independent of which 
descriptions are received. We focused on a special case of the symmetric MD prob- 
lem where only a single parameter (i.e. N) controls the redundancy tradeoffs between 
the central and the side distortions. We showed that the optimal amount of redundancy 
is in independent of the source distribution, the target rate and the type of lattices used 
for the side quantizers. 

The practical design allows an arbitrary number of descriptions and the optimal 
number of descriptions depends (among other factors) upon the packet-loss probabil- 
ity. The theoretical rate-distortion results were proven for the case of K < 3 descrip- 
tions and conjectured to be true in the general case of arbitrary K descriptions. 
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Packet-loss probability [%] 

Figure 5.6: Distortion as a function of the packet-loss probability for the A2 quantizer The 
target entropy is 6 bit/dim., so each description gets 6/K bit/dim. Thick lines show numerical 
performance and thin solid lines show theoretical performance. The two curves at the top 
(coinciding) illustrate the case of K = 1, the two curves in the middle illustrate the case of 
K = 2, and the bottom two curves illustrate the case ofK = 3 descriptions. 




Packet-loss probability [%] 

Figure 5.7: Estimation error as a function of the packet-loss probability for dif- 

ferent number of descriptions. The top curve is for K = 1, the second from the top is for 
K = 2, and so on. The bottom curve is for K = 5. 



Chapter 



/C-Channel Asymmetric Lattice 
Vector Quantization 

In this chapter we will focus on asymmetric MD-LVQ for K > 2 descriptions, see 
Fig. 16. II Asymmetric schemes offer additional flexibility over the symmetric schemes, 
since the bit distribution is also a design parameter and different weights are intro- 
duced in order to control the distortions. In fact, symmetric MD-LVQ is a special case 
of asymmetric MD-LVQ. 

In [27, 28] asymmetric two-channel MD-LVQ systems are derived subject to en- 
tropy constraints on the individual side entropies. However, since these schemes are 
subject to individual side entropy constraints and not subject to a single constraint 
on the sum of the side entropies, the problem of how to distribute a total bit budget 
among the two descriptions is not addressed. In this chapter we derive MD quantizer 
parameters subject to individual side entropy constraints and/or subject to a total en- 
tropy constraint on the sum of the side entropies. We then show that the optimal bit 
distribution among the descriptions is not unique but is in fact characterized by a set 
of solutions, which all lead to minimal expected distortion. 

For the case of K ^ 2 our design admits side distortions which are superior to the 
side distortions of [27,28] while achieving identical central distortion. Specifically, we 
show that the side distortions of our design can be expressed through the dimension- 
less normalized second moment G{Sl) of an L-sphere whereas the side distortions of 
previous asymmetric designs [27, 28] depend on the dimensionless normalized second 
moment G{A) of the i-dimensional lattices. More accurately, the difference in side 
distortions between the two schemes is given by the difference between G{Sl) and 
G(A). Notice that G{Sl) < G(A) with equality for L = I and for L ^ oo by a 
proper choice of lattice [161], cf. Section [3.3. II We also show that, for the case of 
K = 3 and asymptotically in lattice vector dimension, the side distortions can again 
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be expressed through G{Sl) and we further conjecture this to be true fov K > 3 
descriptions. 



X 



Encoder 



Description 



Rq 
Ri 



Description 1 
Description K-i \ Rk-i , 



*' Erasure 
Ciiannel 



Decoder 



■X 



Figure 6.1: General K-channel system. Descriptions are encoded at an entropy of Ri, i = 
. . . ,K — 1. The erasure channel either transmits the i"' description errorless or not at all. 



6.1 Preliminaries 

To be consistent with the previous chapter we will here introduce the set of lattices 
required for the asymmetric design and emphasize how they differ from the symmetric 
design. 

Just as in the symmetric case we use a single lattice Ac as the central quantizer. 
However, we will make use of several sublattices Aj, i = 0, . . . , — 1 for the side 
quantizers. In fact, we use one side quantizer (sublattice) for each description. We 
assume that all sublattices are geometrically-similar to Ac and clean. The sublattice 
index of the i*'* sublattice A^ is given by Ni = |Ac/Ai|, Ni G Z"*". The volume Vi 
of a sublattice Voronoi cell in the i*'* sublattice is given by Vi = NiV, where v is the 
volume of a Voronoi cell of Ac. As in the synmietric case we will also here make 
use of a product lattice A^ C A, C Ac of index TV^r = \^cI^tv \ in the design of the 
index-assignment map. 

The general framework of asymmetric MD-LVQ is similar to the symmetric case. 
We use a single index-assignment map a, which maps central lattice points to K- 
tuples of sublattice points. The main difference is that in the asymmetric case the 
sublattice index values, A^^, i = 0, . . . , if — 1, are not necessarily equal, which means 
that the side descriptions rates Ri are not necessarily equal either. Furthermore, the 
weights for the case of receiving k out of K descriptions depend upon which k de- 
scriptions are considered. This was not so in the synmietric case. 



6.1.1 Index Assignments 

The index assignment map (or labeling function) differs from the symmetric case in 
that it maps from a single lattice to several distinct sublattices. Specifically, let a de- 
note the labeling function and let the individual component functions of a be denoted 
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by tti. The injective map a that maps Ac into Ao x • • • x Ak-i, is then given by 

a{Xc) = (Q;o(Ac),ai(Ac), . . . , a_R-_i(Ac)) (6.1) 
- (Ao,Ai,...,Ak-i), (6.2) 

where ai{\c) = Ai £ Aj and i — 0, . . . , K — 1. 

We generaHze the approach of the previous chapter and construct a product lattice 
Att which has N^^ central lattice points and Nt^/Ni sublattice points from the j*'' 
sublattice in each of its Voronoi cells. The Voronoi cells V-^ of the product lattice 
Ajr are all similar so by concentrating on labeling only central lattice points within 
one cell, the rest of the central lattice points may be labeled simply by translating 
this cell throughout M^. Without loss of generality we let — Y[f=o^ i-^- ^ 
construction we let A^r be a geometrically-similar and clean sublattice of A,; for all 
With this choice of A^^, we only label central lattice points within V^(0), which is the 
Voronoi cell of Att around the origin. 



6.1.2 Rate and Distortion Results 

The central distortion Dc is identical to that of a symmetric system, which is given 
by ( 15.13b . It also follows from the symmetric case see ( 15.141) that the side distortion 
for the i*^ description is given by 

D,^D, + ^^ J2 \\Xc-a,{Xc)f, i^O,...,K-l. (6.3) 

" Aeev,(o) 

Definition 6.1.1. Ri denotes the entropy of the individual descriptions. The entropy 
of the i*'' description is defined as Ri = H{ai{Q{X))) / L. 

The side descriptions are based on a coarser lattice obtained by scaling the Voronoi 
cells of the central lattice by a factor of A'^;. Assuming the pdf of X is roughly constant 
within a sublattice cell, the entropies of the side descriptions are given by 

R,^h{X)-^\og^{N,v). (6.4) 

The entropies of the side descriptions are related to the entropy Rc of the central 
quantizer, given by ( 15.181 ), by 

R^~ Rc-^\og^{N,). 



■^''From Lemma l2.3.1l it follows that the product of admissible index values leads to an admissible index 
value. 
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6.2 Construction of Labeling Function 

In this section we construct the index-assignment map a, which takes a single vector 
Ac and maps it to a set of K vectors {A;}, i ~ 0, . . . , K — 1, where A; € A^. The 
mapping is invertible so that we have Ac = a~^(Ao, . . . , Xk-i)- 

In asymmetric MD-LVQ weights are introduced in order to control the amount of 
asymmetry between the side distortions. We will in the following assume that these 
weights are based on the packet-loss probabilities of the individual descriptions. How- 
ever, it should be clear that the weights are not limited to represent packet-loss prob- 
abilities but can in fact be almost arbitrarily chosen. We will consider the case where 
the index-assignment map is constructed such that the expected distortion, given by 
the sum of the distortions due to all possible description losses weighted by their cor- 
responding loss probabilities, is minimized. 

In addition to knowing the weighted distortion over all description losses it is also 
interesting to know the distortion of any subset of the K descriptions. This issue is 
considered in Section l63] 

6.2.1 Expected Distortion 

At the receiving side, X e is reconstructed to a quality that is determined by 
the received descriptions. If no descriptions are received we reconstruct using the 
expected value, EX, and if all K descriptions are received we reconstruct using the 
inverse map a~^{Xo, . . . , Xk-i), hence obtaining the quality of the central quantizer. 
In all other cases, we reconstruct to the average of the received descriptions as was 
done in the symmetric case. 

There are in general several ways of receiving k, out of K descriptions. Let 
denote an index set consisting of all possible k combinations out of {0, . . . , X — 1} so 
that [^(■^■"'l = (^). We denote an element of by I = {Iq, L-i}. The 

complement F of / denotes the K — k indices not in /, i.e. F — {0, . . . ,K — 
We will use the notation ^^^''^^ to indicate the set of all I £ that contains 

the index i, i.e., = {I : I e ^(■^.«) and i e 1} and similarly ^[f'""^ = 

{I : I and i,j e I}. Furthermore, let be the packet-loss probability 

for the i*^ description and let fii = 1 — pi he the probability that the z*'* description 
is received. Finally, let p(0 = Dje/ Ilje/^ Pj- -PI-^^^'"'') = J2ieJ^(K,^) p{l), 
= I]/gj^.<^f>») -P(0 andp(^-^^'''') = J^i^j^'-'^."' PiO- For example, for 
ii: 3 and K = 2 we have ^(^,2) ^ ||q^ ;^|^ 3}, {1, 2}} and hence p(^(3.2)) ^ 
fJ-ofJ'iP2 + A'oM2Pi + MiM2Po- In a similar manner for K = 6 and k = 3 we have 

■^1,2'^ = {{0, 1, 2}, {1, 2, 3}, {1, 2, 4}, {1, 2, 5}}, 

and 

Pi-^uf^) = M0M1M2P3P4P5 + M1M2M3P0P4P5 + M1M2M4P0P3P5 + MlM2M5P0P3P4- 
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As in the symmetric case, upon reception of any k out of K descriptions we re- 
construct to X using 

The distortion when receiving a set of descriptions can be derived in a similar way 
as was done in the symmetric case. Thus, by use of (|5.14t and ( 16. 3t it can be 
shown that the norm of ( |6.3l l. when receiving descriptions i and j, should read 
||Ac — 0.5(Q!i(Ac) + Q!j(Ac))|p. It follows that the expected distortion when receiving 
K out of K descriptions is given by 



1 1 



E 



^ K— 1 
K ^ ^ 



(6.5) 



LN; 



K — 1 



where A/ . ~ ai- (Ac) and the two special cases k e {0, K} are given by 



iK,0) 



{K,K) 



6.2.2 Cost Functional 



From ( 16.51 1 we see that the distortion Da ''^ may be split into two terms, one describ- 
ing the distortion occurring when the central quantizer is used on the source, and one 
that describes the distortion due to the index assignment. An optimal index assign- 
ment minimizes the second term in ( 16.51 ) for all possible combinations of descriptions. 
The cost functional J*^^) to be minimized by the index-assignment algorithm can then 
be written as 



jiK) 



K-1 

K=l 



where 



-. K — l 



(6.6) 



(6.7) 



The cost functional should be minimized subject to some entropy-constraints on the 
side descriptions or on e.g. the sum of the side entropies. We remark here that the side 
entropies depend solely on i/ and iV^ (see i6.4i ) but not on the particular choice of 
if-tuples. In other words, for fixed Ni's and a fixed ly the index assignment problem 
is solved if (I6.6I 1 is minimized. The problem of choosing ly and iV^ such that certain 
entropy constraints are not violated is independent of the assignment problem and 
deferred to Sectionl64l 
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Theorem 6.2.1. For any 1 < k < X we have 



E E ^(0 



Ac--E^'. 

Ac ^ 

^ A'-2 7<-l 
72 E E 



^(/^,K)^ E P 



p(rf-'=0|l|A.-A,|| 



)A, 



i=0 \ 

Proof. See Appendix ll.il 

The cost functional (16.6b can by use of Theorem l6.2.1l be written as 



□ 



Acey„(o) 

^-2 K-l 



K-l 



4=0 



+ ^E E 11^' -^. 



i=0 j=i+l 



(6.8) 



The first term in (I6.8I 1 describes the distance from a central lattice point to the weighted 
centroid of its associated X-tuple. The second term describes the weighted sum of 
pairwise squared distances (WSPSD) between elements of the iiT-tuple. In Section l63] 
(Proposition l6 .3.2\ we show that, under a high-resolution assumption, the second term 
in ( I6.8I 1 is dominant, from which we conclude that in order to minimize ( 16.6b we must 
use A'-tuples with the smallest WSPSD. These /^-tuples are then assigned to central 
lattice points in such a way, that the first term in ( 16. 8b is minimized. 



6.2.3 Minimizing Cost Functional 

We follow the approach of the symmetric case and center a region V at all sublattice 
points Aq e Ao n T4-(0), and construct X-tuples by combining sublattice points from 
the other sublattices (i.e. A;, i ~ 1, . . . ,K — 1) within V{\q) in all possible ways 
and select the ones that minimize ( 16.6b . For each Aq G Aq n K-(O) it is possible 
to construct Hi^T^ iiT-tuples, where Ni is the number of sublattice points from 
the z*'' sublattice within the region V. This gives a total of (Nt^/Nq) Yl^^^ Ni K- 
tuples when all Ap G Aq n (0) are used. The number Ni of lattice points within a 
connected region V of may be approximated hy Ni ~ vjvi where v is the volume 
of V , which is an approximation that becomes exact as the number of shells of the 
lattice within V goes to infinity, cf. Section 15.3.31 Therefore, our analysis is only 
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exact in the asymptotic case of Ni ^ oo and Vi — > 0. Since Ni w v/vNi and we 
need A^o -ftT-tuples for each Aq S V^7r(0) we see that 

K-l ^K-l K-l 

^0 = n - ^ n 

1=1 i=i 

so in order to obtain at least A'o iC-tuples, the volume of V must satisfy 

K-l 

lim v>vT\nI'''-^^^\ (6.9) 

For the symmetric case, i.e. N ~ Ni, i — 0, . . . , K — 1, we have i> > vN-^^^-^^^\ 
which is in agreement with the results obtained in Chapter|5] 

Before we outline the design procedure for constructing an optimal index assign- 
ment we remark that in order to minimize the WSPSD between a fixed and the set 
of points {Xj e Aj n V} it is required that V forms a sphere centered at A^. The 
design procedure can be outlined as follows: 

1. Center a sphere V at each Aq G Aq H V^7r(0) and construct all possible K- 
tuples (Ao, Ai, . . . , Aa'-i) where A; G A.; n V{Xo) and i = 1, . . . , K — 1. This 
ensures that all iC-tuples have their first coordinate (Ao) inside ^-(0) and they 
are therefore shift-invariant. We will only use if -tuples whose elements satisfy 
li -^i ^ -^j II < Vi, j e 0, . . . if — 1, where r is the radius of V. Make V large 
enough so at least A^o distinct iiT-tuples are found for each Aq. 

2. Construct cosets of each if-tuple. 

3. The central lattice points in Ac n Vtt (0) must now be matched to distinct K- 
tuples. As in the symmetric case, this is a standard linear assignment problem 
[151] where only one member from each coset is (allowed to be) matched to a 
central lattice point in 14^(0). 

The restriction ||A.i — Aj|| < r from step 1), which is used to avoid bias towards 
any of the sublattices, reduces the number of distinct if-tuples that can be constructed 
within the region V. To be able to form A*o if-tuples it is therefore necessary to use a 
region V with a volume larger than the lower bound in ( I6.9l l. In order to theoretically 
describe the performance of the quantizers we need to know the optimal D, i.e. the 
smallest volume which (asymptotically for large Ni) leads to exactly A'o iiT-tuples. 
In Section 1533] a dimensionless expansion factor ijj]^ which only depends on K and 
L was introduced, tp]^ was used to describe how much V had to be expanded from 
the theoretical lower bound (16. 9t , to make sure that N^ optimal if-tuples could be 
constructed by combining sublattice points within a region V. 

Lemma 6.2.1. The dimensionless expansion factor V'l for the asymmetric case is 
identical to the one for the symmetric case. 
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Proof. Follows by replacing the constant Vg by Vi in the proof of Theorem l5.3.2l 

□ 

Adopting this approach leads to 

Remark 6.2.1. It might appear that the shift invariance restriction enforced by using 
only one member from each coset will unfairly penalize Aq. However, the next two 
lemmas prove that, asymptotically as Ni — > oo, there is no bias towards any of the 
sublattices. We will consider here the case of K > 2 (for K = 2 we can use similar 
arguments as given in [28]). 

Lemma 6.2.2. For K > 2 the number of iiT-tuples that is affected by the coset re- 
striction is (asymptotically as Ni oo, Vi) neglectable compared to the number of 
if-tuples which are not affected. 

Proof. See Appendix|L4l □ 

Lemma 6.2.3. The set of K-taples that is constructed by centering V at each 
e 14(0) n Aq is asymptotically identical to the set constructed by centering V at 
each Aj e 14(0) n A.^, for any i e {1,. . .,K - 1}. 

Proof. See AppendixlL4l □ 

Remark 6.2.2. The X-tuples need to be assigned to central lattice points within 14 (0). 
This is a standard linear assignment problem where a cost measure is minimized. 
However, solutions to linear assignment problems are generally not unique. Therefore, 
there might exist several labelings, which all yield the same cost, but exhibit a different 
amount of asymmetry. To achieve the specified distortions it may then be necessary 
to e.g. use time sharing through a suitable mixing of labelings. 

6.2.4 Comparison to Existing Asymmetric Index Assignments 

In this section we have presented a new design for asymmetric MD-LVQ based on the 
asymmetric design of Diggavi et al. [28]. The main difference between the existing 
design of Diggavi et al. and the proposed design is that of the shape of the region 
within which sublattice points are distributed. More specifically, in the design of 
Diggavi et al., a given sublattice point Aq G Aq is paired with a set of sublattice 
points of Al which are all evenly distributed within a Voronoi cell of A^r, the product 
lattice. However, in the proposed design, a sublattice point Aq G Aq is paired with a 
set of sublattice points of Ai which are all evenly distributed within an L-dimensional 
hypersphere. 

Let us emphasize some of the advantages as well as weaknesses of the proposed 
design. 
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• Advantages 

1. The side distortion is reduced (compared to the previous design) when 
finite dimensional lattice vector quantizers are used (when the dimension 
is strictly greater than one). To see this, notice that the side distortion is 
a function of the dimensionless normalized second moment of the region 
over which the sublattice points are distributed. For i = 1 as well as 
L ^ oo spheres pack space and it is possible to have spherical Voronoi 
cells of by a proper choice of product lattice. 

2. To simplify the design it is often convenient to base the product lattice 
upon the simple hypercubic lattice. In this case, the side distortion 
of the design of Diggavi et al. is independent of the vector dimension 
of the lattices, whereas with the proposed design the distortion steadily 
decreases as the dimension increases. The reduction in side distortion is 
upper bounded by approximately 1 .53 dB per description. 

3. The proposed design scales easily to more than two descriptions. It is not 
clear how to obtain more than two descriptions with the previous designs. 

• Weaknesses 



1. The design of Diggavi et al. exploits several geometric properties of the 
underlying lattices to ensure that any single sublattice point of Aq is paired 
with exactly iVo sublattice points of Ai. On the other hand, the proposed 
design guarantees such a symmetry property only in asymptotic cases. 
Thus, in practice, if such a symmetry property is desired, one might need 
to search within a set of candidate solutions. 



6.3 High-Resolution Analysis 

In this section we derive high-resolution approximations for the expected distortion. 
In line with the high-resolution analysis presented in Chapter |5] we let Ni oo and 
Vi — > 0, i.e. for each sublattice the index increase, while the volume of their Voronoi 
cell shrink. 
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6.3.1 Total Expected Distortion 



Using Theorem 16. 2. II the expected distortion i6.5\ when k out of K descriptions are 
received can be written as 



^ K — 1 



3=0 



K-l 



Xr. 



K-2 K-l / ( cfi{K,K),f ^{K,k). \ 



if-2 K-l 
i=0 i=i+l 



(6.10) 



Proposition 6.3.1. For K = 2 and asymptotically as A'^i — > oo, i/^ ^ as well as 
for K = 3 and asymptotically as Ni, L ^ oo and j/j ^ 0, we have for any pair of 
sublattices, (A^, A^), i,j = 0,...,K—l, i ^ j. 



K-l 



Acev„(o) 
Proof. See Appendix |I.2| 



□ 



Conjecture 6.3.1. Proposition 16 . 3 . 1 1 is true for any K asymptotically as L, Ni 
and i^i 0, Vi. 



Proposition 6.3.2. For Ni ^ oo we have 



E 

Ac e 14(0) 



K-l 



i=0 



0. 



Proof. See Appendix|L 



□ 



By use of Propositions 16.3.11 and 16.3.21 and (15.13b it follows that ( I6.IOI 1 can be 
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written as 

m=0 

where /J^^ '') depends on the packet-loss probabihties and is given by 

i=0 j=i+l \ ^ ^ ' / 

The total expected distortion D^J^^ is obtained by summing over k including the cases 
where k = and k = K, 

(6.11) 



j=0 

where 



K = l 

and 

if 



Using (15.18b and ( 16.41 ) we can write and Ni as a function of differential entropy 
and side entropies, that is 

and 

i=0 

Inserting these results in ( 16.1 11 1 leads to 

Z?i^) « G(A,)22(^(^)-^^)p(^(^)) 

i=0 

(6.12) 

where we see that the distortion due to the side quantizers is independent of the type 
of sublattices. 
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6.4 Optimal Entropy- Constrained Quantizers 

In this section we first derive closed-form expressions for the optimal scaling factors 
v and Ni subject to entropy constraints on the K side descriptions. With these scaling 
factors we are able to construct a central lattice and K sublattices. The index assign- 
ments are then found using the approach outlined in Section 16.21 The central lattice 
and the K side lattices combined with their index assignment map completely specify 
an optimal scheme for asymmetric entropy-constrained MD-LVQ. We then consider 
the situation where the total bit budget is constrained, i.e. we find the optimal scaling 
factors subject to entropy constraints on the sum of the side entropies Ri < R*, 
where R* is the target entropy. We also find the optimal bit distribution among the K 
descriptions. 



6.4.1 Entropy Constraints Per Description 

We assume K descriptions are to be used. Packet-loss probabilities pi,i — 0, . . . , K — 
1, are given as well as entropy-constraints on the side descriptions, i.e. Ri < R*, 
where R* are known target entropies. To be optimal, the entropies of the side descrip- 
tions must be equal to the target entropies, hence by use of ( 16.4b we must have that 

R, = hiX)-^\og^iN,iy)=R*, 

from which we get 

Ar,j.^2^™-^*) =T„ (6.13) 

where are constants. It follows that Ni = Tijv and since Hilo^ jY^2/L(K i) _ 
^-2KlL{K-\) J|^~^ ^VL{K-i) express ( 16.1 It as a function of v, i.e. 

i- 

+ \E\\Xf\{p,^ 



i=0 



i=0 

where r = Jl^Io^ n- 

Differentiating w.r.t. v and equating to zero gives, 

ov L 

L[K - I) 
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from which we obtain the optimal value of v 

L{K-1) 



IK 



2K 



(6.14) 



'X-lG(A,)p(^W)^ 
The optimal A^/s follow easily by use of ( I6.13l l: 



L(K-l) 
IK 



^^ = ^ = "^^ U(^'^)G(5Z)-p?rj ■ ^'-''^ 

Eq. ( 16.151 ) shows that the optimal redundancies A'^^'s are, for fixed K, independent of 
the sublattices. Moreover, since tit^^I^ = ^i '' the source-dependent 

term h{X) is eliminated and it follows that the redundancies A^^ are independent of 
the source but also of actual values of target entropies (Ni depends only upon the 
difference between the average target entropy and i?*). 

6.4.2 Total Entropy Constraint 

First we observe from (I6.12l i that the expected distortion depends upon the sum of 
the side entropies and not the individual side entropies. In order to be optimal it is 
necessary to achieve equality in the entropy constraint, i.e. R* = Ri. From (16.4b 
we have 

K-l ^ K-1 

R*=Y,R^ = Kh{X) -j^Y. ^°?>2{N^v)■ 

i=0 4=0 

This equation can be rewritten as 

lliN,,.) = 2^(^''(^)-«*) = r„ (6.16) 

i=0 

where is constant for fixed target entropy and differential entropies. Writing (I6.I6I 1 
as 



K-l 



j2/L(K-l) ^ ^-2K/L(K-1)^2/HK-1) 



1=0 

and inserting in ( 16.1 It leads to 

+ jE\\xri[p,. 



i=0 
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Differentiating w.r.t. v and equating to zero gives 
oiy L 

- 1) 

from which we obtain the optimal value of v, that is 



r^A--lG(Ae)p(^W) 



2/C 



(6.17) 



We note that this expression is identical to ( 16.141 ). The results of this section show that 
the optimal v is the same whether we optimize subject to entropy constraints on the 
individual side entropies or on the sum of the side entropies as long as the total bit 
budget is the same. 

At this point we still need to find expressions for the optimal Ri (or equivalently 
optimal Ni given v). Let Ri = aiR*, where Oi = 1, > 0, hence R* — Ri. 
From ( 16.41 ) we have 

i?, = h{X) - log^iN.u) = a^R*, 
which can be rewritten as 

N, = jy-i2i(''(^)-i,-R*)^ 
Inserting ( I6.17l i leads to an expression for the optimal index value Ni, that is 



-L(g-l) 



(6.18) 



It follows from ( 15.18b and (16.4b that Rc > aiR* so that a^ < Rc/R*. In addition, 
since the rates must be positive, we obtain the following inequalities: 

0<aiR*<Rc, i = 0,...,K -1. (6.19) 

Thus, when we only have a constraint R* on the sum of the side entropies, the 
individual side entropies Ri = aiR* can be arbitrarily chosen (without loss of perfor- 
mance) as long as they satisfy ( 16.19b and J2i = 1- We remark that Ri is bounded 
away from zero by a positive constant, cf. ( 14.1b and ( 14.21 ). For example, for the two- 
channel case we have Rq = aoR* and Ri = aiR* = (1 — ao)R*, so that Rc > 
{l-aa)R* which implies that R* - Rc < Ro < Rc^ 

This result leads to an interesting observation. Given a single entropy constraint on 
the sum of the side entropies, the optimal bit distribution among the two descriptions 



^Recall that Rc is fixed, since it depends on u wliicli is given by 16. 17t . 
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is not unique but contains in fact a set of solutions (i.e. a set of quantizers) which 
all lead to minimal expected distortion^ This allows for additional constraints to 
be imposed on the quantizers without sacrificing optimality with respect to minimal 
expected distortion. For example, in some mobile wireless environments, it might be 
beneficial to use those quantizers from the set of optimal quantizers that require the 
least amount of power. 

6.4.3 Example With Total Entropy Constraint 

Let us show by an example some interesting aspects resulting from the fact that we 
obtain a set of candidate solutions, which all minimize the expected distortion. For ex- 
ample, consider IP-telephony applications, which with the recent spread of broadband 
networks are being used extensively throughout the world today. More specifically, 
let us consider a packet-switched network where a user has access to two different 
channels both based on the unreliable user datagram protocol [133]. Channel is a 
non priority-based channel whereas channel 1 is a priority-based channel or they are 
both priority-based channels but of different priorities. Equivalently this network can 
be thought of as a packet-switched network where the individual packets are given 
priorities; low or high priority. In any case, we assume that only a single packet is 
transmitted on each channel for each time instance (this can be justified with e.g. tight 
delay constraints). The priority-based channel favor packets with higher priority and 
the packet-loss probability pi on channel 1 is therefore lower than that of channel 
0, i.e. pi < pq. Assume the Internet telephony service provider (ITSP) in question 
charges a fixed amount of say $1 ($2) per bit transmitted via channel (channel 1). 
If we then use say 6 bits on channel 1 the quality is better than if we use the 6 bits 
on channel 0. It is therefore tempting to transmit all the bits through channel 1 (or 
equivalently send both packets with high priority) since it offers better quality than 
channel 0. However, our results reveal that it is often beneficial to make use of both 
channels (or equivalently send two packets simultaneously of low and high priority). 
The importance of exploiting two channels is illustrated in Table l6.l] for the examples 
given above for a total bit budget of 6 bits and packet-loss probabilities po = 5% and 
pi = 2%. Notice the peculiarity that since the total bit budget is limited to 6 bits then 
even if the user is willing to pay more than $8 the performance would be no better 
than what can be achieved when paying exactly $8. The last column of Table |6T| 
describes the expected distortion occurring when quantizing a unit-variance Gaussian 
source which has been scalar quantized at a total entropy of 6 bit/dim. The packet-loss 
probabilities are po — 0.05 and pi — 0.02. The quantization error (hence not taking 

■*'ln retrospect, this is not a surprising result since, for tlie two-description case, we already saw that for 
a fixed distortion tuple (Dc, -Do, Di) the lower bound of the rate region is piece-wise linear, cf. Fig. 14.51 
Furthermore, when the sum rate is minimum, this line segment has a 45 degree (negative) slope. Hence, 
any choice of rate pairs on this line segment satisfies the sum rate. The new observation here, however, is 
that now we have a practical scheme, which for any number of descriptions, also satisfies this property. 
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Network Rq Ri Price Quality Expected distortion 



Single-channel 


6 





$6 


Poor 


-12.98 dB 


Single-channel 





6 


$12 


Good 


-16.91 dB 


Two-channel 


2 


4 


$10 


Optimal 


-22.20 dB 


Two-channel 


4 


2 


$8 


Optimal 


-22.20 dB 



Table 6.1: A total bit budget of 6 bits is spent in four different ways. The bottom row shows the 
most economical way of spending the bits and still achieve optimal performance. The packet- 
loss probabilities are po = 5% and pi = 2%. 



packet losses into account) for an optimal entropy-constrained SD system is —34.59 
dB but the expected distortion is dominated by the estimation error due to description 
losses, i.e. 10 logiQ(po) = -13.01 dB and 10 logio(pi) = -16.99. It follows that the 
expected distortion for channel and channel 1 is given by —12.98 dB and —16.91 
dB, respectively. For the two-description system the expected distortion is found by 
use of (16.12b to be —22.20 dB, hence a gain of more than 5 dB is possible when using 
both channels. 



6.5 Distortion of Subsets of Descriptions 

We have so far considered the expected distortion occurring when all possible combi- 
nations of K descriptions are taken into account. In a sense this corresponds to having 
only a single receiver In this section we consider a generalization to multiple receivers 
that have access to non-identical subsets of the K descriptions and where no packet 
losses occur For example one receiver has access to descriptions {0,3} whereas an- 
other has access to descriptions {0, 1, 2}. A total of 2^ — 1 non-trivial subsets are 
possible. We note that the design of the index-assignment map is assumed unchanged. 
We are still minimizing the cost functional given by (I6.6I 1. The only difference is that 
the weights do not necessarily reflect packet-loss probabilities but can be (almost) ar- 
bitrarily chosen to trade off distortion among different subsets of descriptions. For 
example, in a two-description system it is possible to decrease the distortion of de- 
scription by increasing the distortion of description 1 without affecting the rates. 
The main result of this section is given by Theorem |6.5.1| 

Theorem 6.5.1. The side distortion ') due to reception of descriptions {I}, where 
I € (^.«) for any 1 < n < K < 3 is, asymptotically as L, Ni ^ 00 and Ui 0, 
given by 

i=0 
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where 



jel i=0 j=i+l 

and (2) = for K = 1. 

Proof. See Appendix |I.5l □ 

Conjecture 6.5.1. Theorem l6.5.1l is true for K > 3 as L, Ni ^ 00 and I'i 0. 

Remark 6.5.1. For K ~ 2 Theorem l6.5.1l is true also for finite L0 For X = 3 it 
should be seen as an approximation for finite 

In Theorem l6.5.1l the term w^^ '^ is a weight factor that depends on the particular 
subset of received descriptions. For example for = 2 we let 70 — HoPi and 
71 ~ /iipo then for k = 1 the weights for description and 1 are given by 



(m^^i and u;('■^)^^^, (6.20) 

(70+71)^ (70 + 71) 



which are in agreement with the results obtained for the two-channel system in [28]. 

For K = 3 and k = 1 we let 70 = M0P1P2, 7i = and 72 = f^2PoPi and 

the weight for description is then given by 

^(3,0) ^ 7? + 7I + 7172 
(70 + 71 + 72)^ ' 

whereas for k = 2 we use the notation 701 = fJ,aHiP2, 702 = AtoA*2Pi and 712 = 
/^iM2Po from which we find the weight when receiving description and 1 to be 

^(3,{0,l}) ^ 702 + 7l2 + 702712 

4(701 + 702 + 712)^ ' 

6.5.1 Asymmetric Assigmnent Example 

In this section we illustrate by an example how one can achieve asymmetric distortions 
for the case of if = 2 and the lattices. Let Nq 13 and A^i = 9 so that = 117. 
Thus, within VTr{Q) we have 117 central lattice points, 9 sublattice points of Aq, and 13 
sublattice points of Ai. This is illustrated in Fig. 16.21 We first let the weight raticF^ be 
70/71 = 1 so that the two side distortions are identical. In this case several sublattice 



'"This follows since Proposition 16.3 . 1 l is true for any L for K = 2. 

''^It is in fact possible to find an exact expression for finite L. See Remark rH.4. II 

'''The term weight ratio can be related to the ratio of the side distortions by use of )6.22t and )6.23t . 
Specifically, it can be shown that -Di / Do ~ 7o /7i ■ 
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points of Aq located outside t^- (0) will be used when labeling central lattice points 
inside y,r(0). The solid lines in Fig. 16.21 illustrate the 117 edges that are assigned to 
the 117 central lattice points. 

If we let the weight ratio be 70/71 = 4 we favor Aq over Ai. In this case the edge 
assignments are chosen such that for a given edge, the sublattice point belonging to 
Aq is closer to the central lattice point than the sublattice point belonging to Ai. This 
is illustrated in Fig. 16.31 Notice that in this case the sublattice points of Aq used for 
the edges that labels central lattice points within y^(0) are all located within T^7r(0). 
Furthermore, in order to construct the required 117 edges, sublattice points of Ai at 
greater distance from V^ttIO) need to be used. 

In practice, large index values are required in order to achieve large weight ratios 
70/71 or 71/70- Notice that we can achieve asymmetric side distortions even in the 
case where the sublattices are identical (so that A^o = -^1 and the rates are therefore 
identical) simply by letting 70 7^ 71. Moreover, we can achieve symmetric side distor- 
tions by letting 70 — 71 even when TVo 7^ iVi (i.e. Rq ^ R\). In the case where either 
7o = and 71 7^ or 71 =0 and 70 7^ the scheme degenerates to a successive 
refinement scheme, where the side distortion corresponding to the zero weight cannot 
be controlled. In practice this happens if either 70 3> 71 or 71 3> 70- 
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6.6 Numerical Results 

To verify theoretical results we present in this section experimental results obtained by 
computer simulations. In all simulations we have used 2 • 10® unit-variance indepen- 
dent Gaussian vectors constructed by blocking an i.i.d. scalar Gaussian process into 
two-dimensional vectors. We first assess the two-channel performance of our scheme. 
This is interesting partly because it is the only case where the complete achievable 
MD rate-distortion region is known and partly because it makes it possible to compare 
to existing schemes. We end this section by showing the expected distortion (I6.12l i in 
an asymmetric setup using three descriptions. 

6.6.1 Assessing Two-Channel Performance 

The side distortions Dq and Di of the two-channel asymmetric MD-LVQ system pre- 
sented in [27, 28] are given by ( 14.59b and (14.60b and the central distortion is given 
by 

w G(A,)22(''(^)-^'^). (6.21) 
The asymmetric scheme presented in this paper satisfies 

Do « _^ll_G(5L)22''(^)2-2(«o+fli-flc)^ (6.22) 
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and 

ho + iir 

and the central distortion is identical to j6.21l i. It follows that the only difference be- 
tween the pair of side distortions {Dq, Di) and (Do 7 Di) is that the former depends 
upon G(A^) and the latter upon G{Sl). In other words, the only difference in dis- 
tortion between the schemes is the difference between G{Sl) and G'(A7r). For the 
two dimensional case it is known that 0(82) — I/Att whereas if Att is similar to 
we have G(A7r) = 1/12 which is approximately 0.2 dB worse than G{S2)- Fig. 16.41 
shows the performance when quantizing a unit-variance Gaussian source using the 
quantizer for the design of [27, 28] as well as for the proposed system. In this setup 
we have fixed Rq ~ 5 bit/dim. but i?i is varied in the range 5 - 5.45 bit/dim. To do so 
we fix A^i = 101 and let Nq step through the following sequence of admissible index 
values: 

{101, 109, 113, 117, 121, 125, 137, 145, 149, 153, 157, 169, 173, 181, 185}, 

and for each Nq we scale v such that Rq remains constant. When Nq = 101 then 
Rq = Ri = 5 bit/dim. whereas when A^'o > A^i then i?i > Rq. We have fixed 
the ratio 70/71 = 1-55 and we keep the side distortions fixed and change the central 
distortion. Since the central distortion is the same for the two schemes we have not 
shown it. Notice that Dq (resp. Di) is strictly smaller (about 0.2 dB) than Dq (resp. 
Di). This is to be expected since 0(82) is approximately 0.2 dB smaller than G{A-^). 

6.6.2 Three Channel Performance 

In this setup we let ipL = 1 4808 and the packet-loss probabilities are fixed at pq — 
2.5%, pi = 7.5% except forp2 which is varied in the range [1, 10]%. As p2 is varied 
we update ly according to (16.17b and pick the index values Ni such that ^ ■ Ri < R* . 
Since index values are restricted to a certain set of integers, cf. Section 12.3.11 the 
side entropies might not sum exactly to R* . To make sure the target entropy is met 
with equality we then re-scale as = 2^(''*^^-'~'t^ ^ Ililo^ ■ We see from 

Fig. 16.51 a good correspondence between the theoretically and numerically obtained 
results. 

6.7 Conclusion 

We presented a design for high-resolution A'-channel asymmetric MD-LVQ. Along 
the fines of tfie previous chapter, closed-form expressions for the optimal central and 
side quantizers based on packet-loss probabilities and subject to target entropy con- 
straints were derived and practical quantizers were constructed to verify theoretical 
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Figure 6.4: The side distortions are here kept fixed as the rate is increased. Notice that the nu- 
merically obtained side distortions Do and Di (crosses) are strictly smaller than the theoretical 
Do and Di (thin lines). 



results. For the two-channel case we compared the proposed MD-LVQ scheme to a 
state-of-the-art two-channel asymmetric scheme and showed that the performance of 
the central quantizer was equivalent to that of the state-of-the-art scheme whereas the 
side quantizers were strictly superior in finite dimensions greater than one. The prob- 
lem of distributing bits among the K descriptions was analyzed and it was shown that 
the optimal solution was not unique. In fact, it turned out that bits could be almost 
arbitrarily distributed among the K descriptions without loss of performance. As was 
the case for the symmetric design, the practical design of asymmetric MD-LVQ allows 
an arbitrary number of descriptions but the theoretical rate-distortion results were only 
proven for the case of K < 3 descriptions and conjectured to be true in the general 
case of arbitrary K descriptions. 
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Figure 6.5: Expected distortion as a function of packet-loss probabilities for K = 3 packets 
and an entropy of 3 bit/dim. per description. The packet-loss probabilities are po = 2.5%, pi = 
7.5%, 1% <P2< 10% and ipL = 1.14808. 



Chapter 



Comparison to Existing 
High-Resolution IVID Results 

In this chapter we compare the rate-distortion performance of the proposed MD-LVQ 
scheme to that of existing state-of-the-art schemes as well as to known information 
theoretic high-resolution X-channel MD rate-distortion bounds. 

7.1 Two-Channel Performance 

We will first consider the symmetric case and show that, while the proposed design is 
different than the design of Vaishampayan et al. [139], the two-channel performance 
is, in fact, identical to the results of [139]. Then we consider the asymmetric case and 
show that the asymmetric distortion product given by Lemma l4.1.1l is achievable. 

7.1.1 Symmetric Case 

Let i^T = 2 so that ipj^ = 1. From Theorem l6.5.1l (see also (15.34b ) we see that the side 
distortion (i.e. for fc = 1) for the symmetric case, i.e. Dq — Di and Rg — Ri,i ~ 0,1, 
is given by (asymptotically as N ^ oo and i^s 0) 

Do - ^G{Sl)N^/''i^^^''. (7.1) 

In order to trade off the side rate for the central rate we use an idea of [139] and let 
2-2afl, ^ 4jY-2/L ^ijgj-g < a < 1, which impHes that 

^^2^(°^-+i). (7.2) 
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Let us insert ( I7.2l l into ( 15.191 ) in order to express v asa function of Rs and a, 

J, ^ 2^(''(^)-"-^=--"--i). (7.3) 

From ( |6.21t we know that the two-channel central distortion D,, is given by — 
G(Ac)t^^/^ which by use of i7.3i can be rewritten as 

= G(A,)22(''(-^)-''«=-«=-i\ (7.4) 

which leads to 

^lim Z?,22«»(i+'^) = iG(A,)22''W. (7.5) 
By inserting ( 17.2b and ( 17.3b in ( 17.1b we find 

D„ = lG(^i)24(-«=+i)+2(''W--«-«-i), (7.6) 

which leads to 

lim 002^^^^^-"^ = G{Sl)2^''(^\ (7.7) 

Rs — >oo 

Comparing (17.5b and (17.7b with those of Vaishampayan ( 14.56b and ( 14.57b reveals that 
the performance of the proposed two-channel design achieve the same performance 
as the two-channel design of Vaishampayan et al. [139]. Furthermore, let & = 1 and 
i — > oo and notice that in the memoryless Gaussian case G(5oo)2^''*^^-' — <t\ so that, 
by comparing ( 17.5b and (17.7b with ( 14.1 lb and (14.10b . we see that the high-resolution 
two-channel symmetric rate-distortion function of Ozarow can be achieved. 

Remark 7.1.1. It is important to see that a in (17.5b and (17.7b is bounded away from 
zero and one. In the extreme case where = the ratio of side distortion over 
central distortion is small and N cannot be made arbitrarily large as is required for 
the asymptotic expressions to be valid. On the other hand, when a = 1 we can no 
longer force the cells of the side quantizers to be small compared to the variance of 
the source and the high resolution assumptions are therefore not satisfied. This is also 
true for the general case of if > 2 descriptions. 

Remark 7.1.2. We would like to point out an error in [102] where we overlooked the 
requirement that a < 1. In [102] we showed that the high resolution performance 
of (3, 2) SCECs can be achieved by use of lattice codebooks and index assigments 
(which is true) but we also wrongly claimed that in the extreme case where a = 1, 
lattice codebooks achieve rate-distortion points that cannot be achieved by random 
codebooks, obviously, this cannot be true since, for a = 1, the high resolution assump- 
tions are not satisfied (Remark l7.1.1b . 

7.1.2 Asymmetric Case 

We already showed in Section l63] that the performance of the asymmetric two-channel 
scheme by Diggavi et al. [27,28] can be achieved. In fact, in finite dimensions greater 
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than one, the performance of the proposed scheme was strictly superior to that of 
Diggavi et al. Furthermore, it is easy to show that the high resolution asymmetric 
distortion product presented by Lemma l4. 1 . ll can be achieved. To see this note that by 
use of i622\ , i623[ and dOTI ) we get 



D,{Do + Di + 2^DoDi) = G(A,)22(''(^)-^^) (7.8) 

,,2 I ^,2 



f 7o +7l ^^g^-^2'^hiX)2-2(Ro+R.i-B^a) 

V(7o +7i)2 

}. — iM. — c(Sr V2'iHX)2-i{Ro+Ri'Ro)\ 

V (70+71)^ ^ ^ ; 



+ 2^ 

= G(A,)G(5L)24'*(^)2-2(«o+fli)^ 
which, asymptotically as i ^ 00, leads to Lemma l4.1.1l 



7.2 Achieving Rate-Distortion Region of (3, 1) SCECs 

We will now consider the symmetric three-channel case and show that the rate- 
distortion performance of (3, 1) SCECs can be achieved at high resolution. 

We are interested in the three-channel case, i.e. K = 3, and in the limit of L cx3 
so that 

GiS,) ^ ^ (7.9) 



and 



i^l = \h:- (7.10) 



Furthermore, without any loss of generality, we assume that the source has unit vari- 
ance. Thus, from ( |5.34t we see that 



since Rc = Bs + log2(iV'), 



D^^'^'> = (7.11) 
3 



D(^'''> = (7.12) 



and the central distortion Dc — D^"^'^^ given by ( 16.211 ) can be written as 



2 



D^'-'^=[^) S-^''^- (7-13) 

The following lemma shows that symmetric three-channel MD-LVQ can achieve 
the rate-distortion region of (3,1) SCECs at high resolution. 
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Lemma 7.2.1. At high resolution, the one, two and three-channel distortions of (3, 1) 
SCECs are identical to ( 17.1 lb - (17.131 1 in the quadratic Gaussian case. 

Proof. See Appendix lJ.il □ 



Remark 7.2.1. The notion of a large sublattice index N in i^-channel MD-LVQ cor- 
responds to a large (negative) codebook correlation pq for (if, 1) SCECs and in the 
limit of iV ^ cx) we actually have pq —1/{K — 1). Thus, for if = 3 we have 

^ — 1/2 as iV ^ oo. 



7.3 Achieving Rate-Distortion Region of (3, 2) SCECs 

We will now show that the rate-distortion performance of (3, 2) SCECs can be 
achieved by extending the proposed design of three-channel MD-LVQ to include ran- 
dom binning on the side codebooks. Specifically, we show that the achievable two- 
channel versus three-channel distortion region of (3, 2) SCECs for the memoryless 
Gaussian source and MSE can be achieved under high-resolution assumptions. Since 
the performance of a (3, 3) SCEC is identical to that of a single description scheme, it 
is clear that we can also achieve such performance simply by letting K ~ 1 and only 
use the central quantizer. Explicit bounds for Kyi descriptions were not derived 
in [1 1 1, 1 14] but we expect that these (non-derived) bounds are also achievable with 
the proposed i^T-channel MD-LVQ scheme. 

We will begin by considering the general situation where we allow finite dimensio- 
nal lattice vector quantizers and asymmetric rates and distortions. Then, at the end of 
the section, we focus on the symmetric case and infinite-dimensional lattice vector 
quantizers in order to compare the performance to the existing bounds. 

Recall that the proposed design of if-channel MD-LVQ is able to vary the redun- 
dancy by changing the number of descriptions K as well as the index values Ni. In 
addition, it is possible to trade off distortion among subsets of descriptions, without 
affecting the rates, simply by varying the weights. Increasing Ni and at the same 
time decreasing v so that Vi = NiV remains constant does not affect the rate Ri. 
However, the distortion due to the i^^ description is affected (unless counteracted by 
the weights). For example in the symmetric setup where N = Ni for all i and the 
weights are also balanced, the side distortion due to reception of only a subset of 
descriptions is increased as N is increased and vN is kept constant. However, in this 
case, the central distortion due to reception of all descriptions is decreased. In other 
words, in the symmetric case, for a given K, the degree of redundancy is controled by 
the single parameter N . 
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7.3.1 Random Binning on Side Codebooks of MD-LVQ Schemes 

In order to achieve the performance of general {K, k) SCECs we need to introduce 
more controlUng parameters into the design of if -channel MD-LVQ. To do so we 
follow an idea of Pradhan et al. [Ill] and exploit recent results on distributed source 
coding. More specifically, we apply random binning on the side codebooks of the K- 
channel MD-LVQ scheme. This corresponds in some sense to replacing the random 
codebooks of {K, k) SCECs with structured lattice codebooks except that we also 
have a central quantizer and an index assignment map to consider 

Random binning is usually applied on (in principle infinite-dimensional) random 
codebooks. The idea is to exploit the fact that for a given codevector, say Aq, of 
codebook '^^q, only a small set of the codevectors in codebook "ifi is jointly typical 
with Aq. Then by randomly distributing the codevectors of over Mi bins, it is 
unlikely that two or more codevectors, which are all jointly typical with Aq, end up 
in the same bin (at least this is true if AIi is large enough). Thus, if the binning rate 
^6,1 = log2(A/i) is less than the codebook rate i?i then it is possible to reduce the 
description rate by sending the bin indices instead of the codevector indices. 

The rate and distortion performance of lattice vector quantizers are often described 
using high-resolution assumptions, i.e. the rate of the quantizer is sufficiently high and 
the source pdf sufficiently smooth, so that the pdf can be considered constant within 
Voronoi regions of the code vectors. Under these assumptions the theoretical perfor- 
mance of lattice vector quantizers can be derived for arbitrary vector dimension. This 
is in contrast to the asymptotics used when deriving theoretical expressions for the per- 
formance of random codebooks. For random codebooks the theoretical performance 
is usually derived based on asymptotically high vector dimension but arbitrary rates. 
The theory behind random binning relies upon asymptotically high vector dimension 
and as such when using random binning in X-channel MD-LVQ we make use of both 
asymptotics, i.e. high vector dimension and high rates. It is also worth mentioning 
that we consider memoryless sources with infinite alphabets such as e.g. the Gaussian 
source, whereas the analysis of SCECs relies upon strong typicality and as such only 
discrete alphabet memoryless sources are validP'1 

In lattice codebooks, the code vectors are generally not jointly typical and the 
concept of random binning is therefore not directly applicable. It is, however, possible 
to simulate joint typicality by for example some distance metric, so that code vectors 
close together (in e.g. Euclidean sense) are said to be "jointly typical". The index 
assignments of MD-LVQ is another example of how to simulate joint typicality. We 
use the term admissible -tuple for any set of K code vectors (Aq, . . . , Xk-i) which 
is obtained by applying the index-assignment map on a code vector Ac, i.e. a{Xc) — 
(Aq, . . . , Xk-i) for all Ac G Ac. So for lattice code vectors we exploit that only a 
subset of all if-tuples are admissible iC-tuples which, in some sense, corresponds to 

^"However, in [111] the authors remark that the analysis of SCECs can be generalized to continuous- 
alphabet memoi'yless sources by using the techniques of [41, Ch.7]. 
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the fact that only a subset of all iC-tuples of random code vectors are jointly typical. 

Let J C {0, . . . ,K—1} denote an index set, where | J| — k. A fc-tuple is a set of k 
elements {Xj},j G J where Xj G Aj. The fc-tuple given by {Xj} — {aj{Xc)},j G J 
for any Ac G Ac is said to be an admissible fc-tuple. Each lattice Ai contains an infinite 
number of lattice points (or reproduction vectors) but we show by Lemma 17.3.1 1 that 
only finite sets of these points are needed for the codebooks of the side quantizers and 
we denote these sets by '■^i C A^, where ["^il < oo. The set 

{A.|{A„Ao}}^ ^7 14) 

{A2 G A2 : A2 = a2(Ac) and (ai(Ac), ao(Ac)) = (Ai, Aq), VAc G A^, 

denotes the set of A2's which are in admissible fc-tuples that also contain the specific 
elements Ao and Ai. 

Since we consider the asymmetric case some ambiguity is present in the term 
]j(K,k}^ because it is not specified which k out of the K descriptions that are to be 
considered. To overcome this technicality we introduce the notation D^^''^\ J G J^^, 
where ^ denotes the set of combinations of descriptions for which the distortion is 
specified. For example, if we are only interested in the distortion when receiving de- 
scriptions {0, 1}, {0, 2} or {0, 1, 2} out of all subset of {0, 1, 2}, then = {{0, 1}, 
{0, 2}, {0, 1, 2}} and nothing is guaranteed upon reception of either a single descrip- 
tion or the pair of descriptions {1, 2}. 

We will now outline the construction of {K, J^) MD-LVQ. It can be seen that the 
construction of {K, Jf) MD-LVQ resembles the construction of {K, k) SCECs given 
in[lll]. 



Construction of lattice codebooks Construct a if -channel MD-LVQ system with 
one central quantizer and K side quantizers of rate Ri. Let '^c be the codebook of 
the central quantizer and let Ac(jc) G "^c denote the j*J^ element of "^c- Similarly, 
let where i = 0, ... if — 1 denote the codebook of the j*'' side quantizer and let 
Xi{ji) G denote the j-'* codeword of '^i. Finally, let a be the index-assignment 
function that maps central lattice points to sublattice points. 



Random binning Perform random binning on each of the side codebooks to 
reduce the side description rate from Ri to i?b ^ bit/dim., where we assume Ri > 
Rb,i- Let — 2^''^'~^^-^^""-'^ where 7^ > 0. Assign codewords to each of the 
2LRb,i i^jjjg Qf q^q\i codebook. The codewords for a given bin of codebook "^q is 
found by randomly extracting codewords from '^q uniformly, independently and 
with replacement. This procedure is then repeated for all the remaining codebooks 
%,i=l,...,K -I. 

Encoding Given a source word X G M^, find the closest element Ac G 'rfc and use 
a to obtain the corresponding if -tuple, i.e. Q!(Ac) = {Xo{jo), ■ ■ ■ , Xk-iUk-i))- If 
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the codeword ^ then set ji equal to a fixed special symboO say ji = d. For 
i = 0, . . . , K — 1 define the function fi{Xi{ji)) which indicates the index of a bin 
containing the codeword Xi{ji). If Xi{ji) is found in more than one bin, set fi{\i{ji)) 
equal to the least index of these bins. If is not in any bin, set fi{Xi{ji)) = 

The bin index fi{Xi{ji)) is sent over channel i. 

Decoding The decoder receives some m bin indices and searches through the corre- 
sponding bins to identify a unique admissible m- tuple. 

Remark 7.3.1. For {K, ,J€) MD-LVQ the notion of large block length, i.e. L ^ oo, 
is introduced in order to make sure that standard binning arguments can be applied. 
However, it should be clear that the quantizer dimension is allowed to be finite. If 
finite quantizer dimension is used it must be understood that (finite length) codewords 
from consecutive blocks are concatenated to form an L-sequence of codewords. The 
dimension of the L-sequence becomes arbitrarily large as L ^ oo, but the quantizer 
dimension remains fixed. As such this will not affect the binning rate but the distortion 
tuple is affected in an obvious way. 

Theorem 7.3.1. Let X £ be a source vector constructed by blocking an ar- 
bitrary i.i.d. source with finite differential entropy into sequences of length L. Let 
JC{0,...,if — 1} and let Aj denote the set of codewords indexed by J. The set 
of decoding functions is denoted gj : j ~^ H^^- Then, under high-resolution 
assumptions, if 

E[piX,gji\j))]<D^''^^\ VJGjr, 
where p{-, •) is the squared-error distortion measure and for all S* C J 

>^7« + ^log2(|{A5|Aj-5}|), (7.15) 

the rate-distortion tuple {Rb.o, ■ ■ • , Rb.(K-i), {^^^''^•'jjejr) is achievable. 

Proof. See Appendix lJ.2l □ 

We have the following corollary for the symmetric case: 

CoroUary 7.3.1 (Symmetric case). Let X e be a source vector constructed by 
blocking an arbitrary i.i.d. source with finite differential entropy into sequences of 
length L. For any J C {O,...,^ — IjletAj denote the set of received codewords 
and let gj : (S'je j ^ be the set of decoding functions. Then, under high- 
resolution assumptions, if 

E[p(X, gj{Xj))] < VJ C {0, . . . , if - 1}, I J| > fc, 

where •) is the squared-error distortion measure and for all C J 

''The rate increase caused by the introduction of the additional symbol i9 is vanishing small for large L. 
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^6 >7 + ^l0g2(|{As|A,7_s}|), 

the tuple (i?b, D^-^'*^), D^^'-'^+i), . . . , Z)'^'^)) is achievable. 



(7.16) 



Proof. Follows immediately from Theorem |7.3.1| 



□ 



7.3.2 Symmetric Case 

To actually apply Theorem |7.3.1| we need to find a set of binning rates — 
0, . . . , K — 1, such that (17.15b is satisfied for all subsets S of J and for all elements 
J of J{f. Let us consider the symmetric case where K — 3 and design a (3, 2) 
MD-LVQ system. We then have J = {io, ii} and it suffices to check the two cases 
where S ^ io and S — {io,ii}- Without loss of generality we assume that zq = 
and ii = 1. The number of distinct Ai's that is paired with a given Ao can be 
approximatecjf^by {ipL\fW')^ , where N' = A^^/^ is the dimension normalized index 
value describing the index (redundancy) per dimension. Let S ~ {0,1} and notice 
that |{As}| < |{Ai|Ao}| • \%)\. Then, asymptotically, as TV oo, it follows that 
|{Ai|Ao}| = ('0L-\//V')^. Let us now bound the codebook cardinality. 

Lemma 7.3.1. Y^i\ — 2^^*. Furthermore, the entropy of the quantizer indices is 
upper bounded by 

Proof. See Appendix lJ. 1 1 □ 

We are now able to find by considering the two cases \S\ — 1,2. For jS"! = 1 
we have from (17.161 1 that 



The dominant i?^ is then given by i?b = max(i?b /, Rbji)- Since ( 17.171 ) and ( 17.181 1 
depends upon N' the dominating binning rate depends upon N'. To resolve this prob- 
lem, we form the inequality Rtji > Rb,i and find that N' < 2"^^" So for 
N' < 2'^^" /i/jI we have Rb — Rb.ii- It is interesting to see that when inserting 
N' ~ 2^^=/?/;| in (17.17b we get Rbj = 7 + Rs- Coincidently, Rbj becomes effec- 
tive when the binning rate Rb is equal to the codebook rate Rg, which violates the 
assumption that Rb > Rg- 

It is clear that if we set Rb equal to the lower bound in ( 17. 18b we get 




(7.17) 



whereas for 151=2 




(7.18) 



Rb=^Rs + ^l0g2{N') + -l0g,{^L) 



^-Recall that this approximation becomes exact as Af — > oo. 

^^For large L there is really no loss by assuming that 2^^» is an integer. 
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from which we can express N' and Rg as functions of each other and Ri,, that is 

iV' = 2^^''-2fi=^-2^ (7.19) 

and 

R, = 2i?fc - log2(^L) - i log2(A^')- (7.20) 

It follows that when varying the redundancy per dimension N', the binning rate Rb 
can be kept constant by adjusting Rg according to ( 17.201 ). 

In order to compare these results to the existing bounds we let L — > oo so that by 
inserting (17.20b in (15.341 1 we get 



(7.21) 



12 

The central distortion (Dc = Df^.s)) MD-LVQ is given by 

^(3,3) ^ 2-2^-, (7.22) 
where Rc — Rs + ^og2{N') which leads to 

R, - 2i?b - log2(i^oo) + I log^iN'). (7.23) 
Inserting (17.231 1 into ( I7.22l i leads to 

^(3,3) ^ ^2-4^^ (7.24) 



Lemma 7.3.2. At high resolution, the two and three-channel distortions of (3, 2) 
SCECs are identical to ( 17.21b and ( 17.24b in the quadratic Gaussian case. 

Proof. See Appendix lJ.il □ 



7.3.3 Asymmetric Case 

For the asymmetric case, A' = 3 and where JT = {{0, 1}, {0, 2}, {1, 2}, {0, 1, 2}}, 
i.e. reconstruction is possible when any two or more descriptions are received, it can 
be shown (similar to the symmetric case) that the binning rate i?b ^ is lower bounded 

by RbA = max{Rb.ij , Rb,i,i ) where 

Rb,^, = l0g2(V^L) + \ log2«) - l0g2(A^;) (7.25) 

and 

Rb,^rr - \ logaCV-L) + \ log2«) " \ ^Og^iN'i) + ^i?., (7.26) 
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where iV; = KN[N!^. 

To see this note that if Xi and both are in the same admissible X-tuple, then Ai 
must be within a sphere V centered at \j . The volume of V is v, which implies that 
the maximum number of distinct Ai points within V is approximately v/vi. In other 
words, 

where the approximation becomes exact for large index values. With this it is easy to 
see that 



R,,.> -\og^{\{K\\,}\) 



1 

^ (7.28) 

« log2(VL) + 2 \og^{KN[N'^) - \og^{N,), 

which is identical to (17.25b . From Theorem 17. 3. II we can also see that the pair-wise 
sum rate must satisfy 

Rb.. + Rb., > ylogadaiKAjlAJI) 

^ (7.29) 

« i?, + log2(VL) + 2 l0g2(iV^iV(iV^) - log2(iV,), 

which can equivalently be expressed as Rt^i + Rtj > 7; log2(|C'j||{Ai|Aj}|) from 
which the individual rate requirements can be found to be given by ( 17.26b . 



7.4 Comparison to i^'- Channel Schemes 

In the asymptotic case of large lattice vector quantizer dimension and under high res- 
olution conditions, we showed in the previous sections that existing MD bounds can 
be achieved. However, it is also of interest to consider the rate-distortion performance 
that can be expected in practical situations. Towards that end we presented some nu- 
merical results obtained through computer simulations in Sections [5 .61 and l676l 

In this section we will compare the theoretical performance of the proposed MD- 
LVQ scheme to existing state-of-the-art X-channel MD schemes [18, 127]. While 
the schemes [18, 127] (as well as the proposed scheme) can be shown to be optimal, 
under certain asymptotic conditions, they are not without their disadvantages when 
used in practical situations. We will, however, refrain from comparing implementation 
specific factors such as computational complexity as well as scalability in dimension, 
description rate and number of descriptions. Such comparisons, although relevant, are 
often highly application specific. 

The above mentioned schemes are all based on LVQ and it is therefore possible to 
compare their theoretical rate-distortion performance when finite-dimensional lattice 
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vector quantizers are used. Recall from Section 14.2.21 that the scheme of Chen et 
al. [18] has a rate loss of {2K — 1) L-dimensional lattice vector quantizers The 
scheme of 0stergaard and Zamir [127] was, for the case of K — 2, shown to have 
a rate loss of only two L-dimensional lattice vector quantizers. While this design 
was shown to permit an arbitrary number of descriptions, the rate loss for K > 2 
descriptions was not assessed. The rate loss of the proposed scheme, on the other hand, 
has a somewhat peculiar form. In the case of two descriptions, the rate loss is given 
by that of two L-dimensional quantizers having spherical Voronoi cells|ff| However, 
in the case of K > 2 descriptions, there is an additional term which influences the rate 
loss. 

7.4.1 Rate Loss of MD-LVQ 

To be able to assess the rate loss of MD-LVQ when using finite-dimensional quantizers 
and more than two descriptions, we let Rf denote the description rate (where the 
subscript / indicates that finite-dimensional quantizers are used). Then the distortion 
when receiving a single description out of K = 3 can be found by use of (15.34b to be 
given by 

^(3,1) ^ iG'(5L)(27re)V'iiV'2-2«/. (7.30) 

Equalizing ( 17. lib and (17.30b reveals that the rate loss (Rf — Rs), for LT = 3, is given 
by (at high resolution) 

Rf -R^ = l log2(G(5L)(27re)) + log2(V'L/^oo). (7.31) 

Since ipL < V'l = V'L (^t least for K = 3) we can upper bound the second term 
of (17311 1 bv 

log2(V'L/^oo) < log2(V'oo) = 0.1038 bit/dim. (7.32) 

Fig. 17. II shows log2(?/'L/'/'oo) for 1 < L < 101 for K = 3 using the values of ipL 
from Table |5.1] 

Remark 7.4. 1. For = 2 we have i/jl = 1, VL, and (17. 31b is true. Furthermore, if the 
A' -channel MD-LVQ scheme is optimal also for K > 3, as we previously conjectured, 
then ( 17.31b is true for any K > 2 (at high resolution). 

It is interesting to observe that both terms in (17.311) are independent of the partic- 
ular type of lattice being used. For example, if the product lattice A — is used 
for the side quantizers, then the rate loss vanishes (it becomes identical to zero) even 

^^In the asymmetric case where comer points of the rate region are desired, the rate loss of [18] is only 
that of K lattice vector quantizers. However, in the symmetric case, source splitting is necessary and there 
is an additional rate loss. 

'^This is true in the symmetric case as well as in the asymmetric case. 
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though G(A) = 1/12C| This is not the case with the other two schemes mentioned 
above, i.e. [18, 127]. Fig. 17. 21 shows the rate loss of the different schemes for K — 3 
descriptions. The lattices used are those of Table [TTI Since we do not have ipL values 
for all even L we have in Fig. 17.21 simply used the average of the two neighboring 
values, that is 




(7.33) 



10 20 30 40 50 60 70 80 90 100 
L 

Figure 7.1: The rate loss due to the term log2(i/'i,/^/'oo) is here expressed in hit/dim. as a 
function of the dimension L. 

7.5 Conclusion 

In the previous two chapters we initially used a single index-assignment map to control 
the redundancy between descriptions. In this chapter we then showed that by use of 
random binning on the side codebooks in addition to the index-assignment map it 
was possible to introduce more rate-distortion controlling parameters into the design. 
While the use of random binning is standard procedure in distributed source coding 
and has previously been applied to MD schemes based on random codebooks as well, 

^^Recall that the central distortion depends upon the type of lattice being used. However, at high reso- 
lution conditions, the index value is large (i.e. — > oo) and as such the central distortion is very small 
compared to the side distortion and can therefore be neglected. 
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Figure 7.2: Rate loss of the different three-channel MD schemes. For comparison we have 
included a lower bound ( thick solid curve ), which corresponds to the sum rate loss of three L- 
dimensional quantizers having spherical Voronoi cells. The horizontal dashed line corresponds 
to the sum rate loss of three lattice vector quantizers with hypercubic cells and the circles 
illustrate the performance of a scheme having a sum rate loss of three optimal lattice vector 
quantizers. 

it appears to be the first time it is used in connection with index-assignment based MD 
schemes. 

We showed that the proposed design of MD-LVQ, asymptotically in rate and lat- 
tice vector quantizer dimension, achieves existing rate-distortion MD bounds in the 
quadratic Gaussian case for two and three descriptions. 

In finite lattice vector quantizer dimensions, we showed that the rate-loss of the 
proposed design is superior to existing state-of-the-art schemes. 



Chapter v_y 

Network Audio Coding 



In this chapter we apply the developed MD coding scheme to the practical problem 
of network audio coding. Specifically, we consider the problem of reliable distribu- 
tion of audio over packet-switched networks such as the Internet or general ad hoc 
networksjf^ Thus, in order to combat (excessive) audio packet losses we choose to 
transmit multiple audio packets. 

Many state-of-the-art audio coding schemes perform time-frequency analysis of 
the source signal, which makes it possible to exploit perceptual models in both the 
time and the frequency domain in order to discard perceptually-irrelevant informa- 
tion. This is done in e.g. MPEG-1 (MP3) [93], MPEG-2 advanced audio coding 
(AAC) [94], Lucent PAC [123] and Ogg Vorbis [134]. The time-frequency analy- 
sis is often done by a transform coder which is applied to blocks of the input signal. 
A common approach is to use the modified discrete cosine transform (MDCT) [90] 
as was done in e.g. MPEG-2 AAC [94], Lucent PAC [123] and Ogg Vorbis [134]. In 
this chapter we combine the MDCT with -channel MD-LVQ in order to obtain a 
perceptual transform coder, which is robust to packet losses. 

MD coding of audio has to the best of the author's knowledge so far only been 
considered for two descriptions [3, 119]. However, here we propose a scheme that is 
able to use an arbitrary number of descriptions without violating the target entropy. 
We show how to distribute the bit budget among the MDCT coefficients and present 
closed-form expressions for the rate and distortion performance of the if-channel 
MD-LVQ system which minimize the expected distortion based on the packet-loss 
probabilities. Theoretical results are verified with numerical computer simulations 
and it is shown that in environments with excessive packet losses it is advantageous 
to use more than two descriptions. We verify the findings that more than two descrip- 
tions are needed by subjective listening tests, which further proves that acceptable 

''Part of the research presented in this chapter represents joint work with O. Niamut. 
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audio quality can be obtained even when the packet-loss rate is as high as 30%. 



8.1 Transform Coding 

In this section we describe the MDCT and we define a perceptual distortion measure 
in the MDCT domain. 



8.1.1 Modified Discrete Cosine Transform 

The MDCT is a modulated lapped transform [90] which is applied on overlapping 
blocks of the input signal. A window of 2M time-domain samples is transformed into 
M MDCT coefficients, whereafter the window is shifted M samples for the next M 
MDCT coefficients to be calculated. 

Given a block s e M^*^ the set of M MDCT coefficients is given by [90] 

1 ^\^\ /(2n + M + l)(2fc + l)7r 



= -7^ 2^ h^Sn COS I — 1, fc = 0,...,M~l, 

^ n— ^ 

(8.1) 

where Xk, hn E M and h is an appropriate analysis window, e.g. the symmetric sine 
window [90] 

/.n = sin((n+l)(^)), n = 0,...,2Af-l. (8.2) 
The inverse MDCT is given by 

[90E 

1 ^4^^ /(2n + M+l)(2k + l)TT\ „ , 

s„ = K^= > Xk cos ^—^ , n = 0, . . . , 2A/r - 1. 



(8.3) 



8.1.2 Perceptual Weighting Function 

On each block a psycho-acoustic analysis is performed which leads to a masking curve 
that describes thresholds in the frequency domain below which distortions are inaudi- 
ble. In our work the masking curve is based on a 2nM-point DFT where n G N and 
the computation of the masking curve is described in detail in [140]. Let us denote 
the masking curve by E. We then define a perceptual weight ^ as the inverse of the 
masking threshold S evaluated at the center frequencies of the MDCT basis functions, 
that is 

^^k=^2nk+l^ fc = 0,...,M-l. (8.4) 

^^Notice that the MDCT is not an invertible transform on a block-by-block basis since 2M samples 
are transformed into only M samples. We therefore use the tilde notation to indicate that, at this point, 
the reconstructed samples Sn are not identical to the original samples Sn. In order to achieve perfect 
reconstruction we need to perform overlap-add of consecutive reconstructed blocks [90]. 
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We require /i to be a multiplicative weight but otherwise arbitrary. We will not 
go into more details about /x except mentioning that we assume it can be efficiently 
encoded at e.g. 4 kpbs as was done in [96]. 



8.1.3 Distortion Measure 

Let X e R^^ denote a random vector proces^ and let x G M*^ be a realization of 
X. By Xk and Xk we denote the fc*'' components of X and x, respectively, and we 
will use S^k to denote the alphabet of Xk- The pdf of X is denoted fx with marginals 

We define a perceptual distortion measure in the MDCT domain between x and a 
quantized version a; of a; to be the single-letter distortion measure given b\F°l 



p(a;,i) = ^ A^fclxfc - ifcp, (8.5) 

fe=0 

where fjLk is given by ( I8.4l l. The expected perceptual distortion follows from ( I8.5l l 
simply by taking the expectation over x, that is 

D(x,x) = — / tJ.k\xk ~ Xk\^ fxk{xk)dxk, (8.6) 
where we remark that /i depends on s through x. 



8.1.4 Transforming Perceptual Distortion Measure to ^2 

For the traditional MSB distortion measure which is also known as the £2 distortion 
measure, it is known that, under high-resolution assumptions, a lattice vector quan- 
tizer is good (even optimal as L ^ 00) for smooth sources, see Chapter [3] The 
MSB distortion measure is used mainly due its mathematical tractability. However, 
in applications involving a human observer it has been noted that distortion measures 
which include some aspects of human auditory perception generally perform better 
than the MSB. A great number of perceptual distortion measures are non-difference 
distortion measures and unfortunately even for simple sources their corresponding 
rate-distortion functions are not known. For example, the perceptual distortion mea- 
sure given by ( 18.6b is an input-weighted MSB (because /i is a function of s), hence it 
is a non-difference distortion measure. 

In certain cases it is possible to derive the rate-distortion functions for general 
sources under non-difference distortion measures. For example, for the Gaussian pro- 
cess with a weighted squared error criterion, where the weights are restricted to be lin- 
ear time-invariant operators, the complete rate-distortion function was found in [118]. 

^'in fact it is tlie output of the MDCT of a random vector process S G R^'*^ . 

'"Strictly speaking tliis is not a single-letter distortion measure since the perceptual weight depends upon 
the entire vector. 
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Other examples include the special case of locally quadratic distortion measures for 
fixed rate vector quantizers and under high-resolution assumptions [43], results which 
are extended to variable-rate vector quantizers in [80, 83]. With regards to the MD 
problem, [84] presents a high-resolution rate-distortion region for smooth sources and 
locally quadratic distortion measures for the case of two descriptions. The case of 
vector sources and more than two descriptions remains unsolved. 

Remark 8.1.1. In the SD case it has been shown that it is sometimes possible to apply a 
function (called a multidimensional compressor) on the source signal in order to trans- 
form it into a domain where a lattice vector quantizer is good. This approach was first 
considered by Bennett in [7] for the case of a scalar compressor followed by uniform 
scalar quantization. The general case of a multidimensional compressor followed by 
lattice vector quantization was considered in [85]. In general an L-dimensional source 
vector X is "compressed" by an invertible mapping F^li Hereafter F{X) is quantized 
by a lattice vector quantizer. To obtain the reconstructed signal X, the inverse map- 
ping F^^ (the expander) is applied, that is 

X^F{-)^Q{-)^F~\-)^X, (8.7) 

where Q denotes a lattice vector quantizer It was shown in [85] that an optimal com- 
pressor F is independent of the source distribution and only depends upon the distor- 
tion measure. However, it was also shown that an optimal compressor does not always 
existsl^ In the MD case, results on optimal compressors are very limited. However, 
it was suggested in [84], that a compressor obtained in a similar way as for the SD 
case, might perform well also in the two-description case for smooth scalar processes. 
Unfortunately, we have been unsuccessful in finding an analytical expression for such 
a vector compressor for our distortion measure ( 18.5b . 

In this chapter we will assume that the decoder has access to the perceptual weight 
II, which makes it possible to exploit fi also at the encoder when quantizing the MDCT 
coefficients. This has been done before by e.g. Edler et al. [31]. In addition, in the 
perceptual MD low delay audio coder presented in [ 11 9] a post filter, which resembles 
the auditory masking curve, was transmitted as side information. The input signal 
was first pre filtered by a perceptual filter which transforms the input signal into a 
perceptual domain that approximates an £2 domain. A lattice vector quantizer is used 
in this domain and at the decoder the signal is reconstructed by applying the post filter. 

We adopt the approach of normalizing the input signal by the perceptual weight. 
First we show that, under a mild assumption on the masking curve, this procedure 
transforms the perceptual distortion measure into an £2 distortion measure. From ( I8.6I 1 

*'The invertible mapping F is for historically reasons called a compressor and said to compress the 
signal. However, F is allowed to be any invertible mapping (also an expander) but we will use the term 
compressor to be consistent with earlier Uterature. 

*^In the scalai' case an optimal compressor always exists for a wide range of distortion measures. 
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we have that 



M-l 



-D(a;,x) = — ^ / i^k\xk- Xk\^fxk{xk)dxk (8.8) 

fc=0 

1 /■ 

E E / A^fck/c - ifc|VxJa;fc)(ia;fc 

1 r 

= mT.J2 , - (8.9) 

k=o J "'^fcnv/ 

where yk = Xk^/JIk, Vk — Xky/]Tk and (a) follows by breaking up the integral into 
disjoint partial integrals over each Voronoi cell Vj of the quantizer. In order to perform 
the necessary variable substitution in the integral given by (18.91) we write 



-, =Xk-, (y^) + JJIk. (8.10) 

dxk dxk 

At this point we enforce the following condition on the masking curve. Within each 
quantization cell, the first derivative of the masking curve with respect to the source 
signal is assumed approximately zero so that from ( |8.10t dxk ~ ^^/i Z/ii-rfyt-Fl Inser- 
ting this in (|8.9l l leads to 

D[x,x) ~ TT E E / \y^~ yk?fxA^k)\/l/nkdyk 



k=0 j 

M-l 



\yk-Vk?fYMdyk. (8.11) 



M-l 



fc=0 



since it can be shown that /y^^ (j/^) — fx^ {xk)\/l/ fik cf. [126, p. 100]. In other words, 
simply by normalizing the input signal x by the root of the input-dependent weight fi, 
the perceptual distortion measure for x is transformed into an £2 distortion measure 
for y. Therefore, when quantizing y, the distortion is approximately the same when 
measuring the ^2-distortion i.e. E\\y — or transforming y and y back into x 

and X, respectively, and measuring the perceptual distortion given by i 



8.1.5 Optimal Bit Distribution 

Each block s leads to AI MDCT coefficients, which we first normalize by y/JI and then 
vector quantize using JC-channel MD-LVQ. Since, the number of coefficients in the 

*'To justify this assumption notice that we can approximate the masking curve by piece-wise flat regions 
(since the masking curve also needs to be coded), which means that small deviations of the source will not 
affect the masking curve. 
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MDCT is quite large, e.g. M = 1024 in our case, it is necessary to split the sequence 
of M coefficients into smaller vectors to make the quantization problem practically 
feasible. Any small number of coefficients can be combined and jointly quantized. 
For example, if the set of M coefficients is split into M' bands (vectors) of length Lk 
where k = 0, . . . , M' — 1 it can be deduced from ( 15.40b that the total distortion is 
given b}0 



M'-l 



fe=o (8.12) 



+ i.:2..^i^G(5,)22W^'=)-«^J2^(«--^'=) + ^£;|in.f, 

Lk 

where we allow the quantizers A^ and the number of packets Kk to vary among the 
M' bands as well as from block to block. For a given target entropy R* we need to 
find the individual entropies Rk for the M' bands, such that J^^^k — R* /K and in 
addition we need to find the entropies Rc^ of the central quantizers. For simplicity 
we assume in the following that the M' bands are of equal dimension L', that similar 
central lattices Ac are used, and that the number of packets K is fixed for all k. 

We now use the fact that ( |5.44t and ( 15.451 1 hold for any bit distribution, hence we 
may insert ( 15.441 ) and (15.451 1 into (I8.12l i which leads to individual distortions given by 




K^GiSL') \K-lKi G(A,) 



L' 



L 



where ao is independent of k and given by 



(8.13) 



*^The distortion over individual normalized MDCT coefficients is additive in the MDCT domain (recall 
that we are using a single-letter distortion measure). However, adding the entropies of a set of MDCT coeffi- 
cients is suboptimal unless the coefficients are independent. Futhermore, the individual MDCT coefficients 
will generally be correlated over consecutive blocks. For example, overlapping blocks of an i.i.d. process 
yields a Markov process. For simplicity, we do not exploit any correlation across blocks nor between the 
vectors of MDCT coeffficients (but only within the vectors). 
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In order to find the optimal bit distribution among the M' bands subject to the entropy 
constraint X]fc=o ^ ^fc = R* / K we take the common approach of turning the con- 
strained optimization problem into an unconstrained problem by introducing a Lagran- 
gian cost functional of the form 

M'-l M'-l 

J= Dk + X Rk- (8.15) 

fe=0 k=0 

Differentiating (18.15b w.rt. Rk leads to 

dJ 



= -21n(2)ao22(''(^'«)--R'») + A. (8.16) 



dRk 

After equating (I8.I6I 1 to zero and solving for Rk we get 

In order to eliminate A we invoke the sum-rate constraint X^feLcT^ ^fc — R* /K and 
get 

E - --log. [^^J + E Mn) - RyK, (8.18) 

k — k — 

from which we obtain 

A = 21n(2)ao2-^(^*/^-^"-"^(^'=». (8.19) 
We can now eliminate A by inserting ( 18.19b into (18.17b . that is 

H^^aiUl^^^^^^ + Mn). (8.20, 

With the simple Lagrangian approach taken here there is no guarantee that the 
entropies Rk given by (18.20b are all non-negative. It might be possible to extend 
the Lagrangian cost functional ( 18.15b by M' additional Lagrangian weights (also 
called "complementary slackness" variables [128]) in order to obtain M' inequality 
constraints making sure that Rk > in addition to the single equality constraint 
X = R* /K. While the resulting problem can be solved using numerical techni- 
ques, it does not appear to lead to a closed-form expression for the individual entropies 
Rk ■ It is not possible either to simply set negative entropies equal to zero since this will 
most hkely violate the constraint J^^k — R* /K. Instead we propose a sequential 
procedure where we begin by considering all M' bands and then one-by-one eliminate 
bands having negative entropies. We assign entropies to each band using ( 18.201 ) and 
then find the one having the largest negative entropy and exclude that one from the 
optimization process. This procedure continues until all entropies are positive or zero 
as shown in Table 18. ll 
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1. 




= {0,...,M'-1} 


2. 


h = 




3. 


c — 


\-y\ 


4. 




= {Rk ■.Rk=c + h(Yk) and Rk < 0, fc € ^} 


5. 


If \£ 


f > then goto 2 and set ^ := J^\j, where Rj < Rk.^k e J' 


6. 


Rk 


_{c + h{Yk), k^J 






1 0, otherwise 



Table 8.1: Bit-allocation algorithm. 



The motivation for this approach is that ultimately we would like the contribution 
of each band to the total distortion to be equal, since they are all approximately equally 
sensitive to distortion after being flattened by the masking curve. However, the nor- 
malized MDCT coefficients in some bands have variances which are smaller than the 
average distortion, hence assigning zero bits to these bands leads to distortions which 
are lower than the average distortion over all bands. Therefore, the bit budget should 
only be distributed among the higher variance components. 

8.2 Robust Transform Coding 

In this work we apply MD-LVQ on the normalized coefficients of an MDCT to obtain 
a desired degree of robustness when transmitting encoded audio over a lossy net- 
work. The encoder and decoder of the complete scheme are shown in Figs. |8.1(a)| 
and |8.1(b)l respectively. In the following we describe how the encoding and decoding 
is performed. 

8.2.1 Encoder 

By a we denote the current block, which has been obtained by blocking the input sig- 
nal into overlapping blocks each containing 1M samples. The M MDCT coefficients 
are obtained by applying an M-channel MDCT on s and is represented by the vec- 
tor X. It is worth mentioning that we allow for the possibility to use a flexible time 
segmentation in order to better match the time-varying nature of typical audio signals, 
cf. [95]. Each block is encoded into K descriptions independent of previous blocks 
in order to avoid that the decoder is unable to successfully reconstruct due to previous 
description losses. 
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Figure 8.1: Encoder and decoder. 



As discussed in Section 18.1.51 it is infeasible to jointly encode the entire set of 
M MDCT coefficients and instead we split x into M' disjoint subsets. The MDCT 
coefficients are then normalized by the perceptual weights fi in order to make sure 
that they are approximately equally sensitive to distortion and moreover to make sure 
that we operate in an £2 domain where it is known that lattice vector quantizers are 
among the set of good quantizers. Based on the differential entropies of the normal- 
ized MDCT coefficients y and the target entropy R* we find the individual entropies 
Rk,k — 0, . . . , M' — 1 by using the algorithm described in Table 18.11 Fig. |8.2(a)| 
shows an example of the distribution of differential entropies h{Y) in a 1024-channel 
MDCT. In this example a 10 sec. audio signal (jazz music) sampled at 48 kHz was 
input to the MDCT. Fig. |8.2(b)| shows the coiTesponding discrete entropies assigned 
to each of the 1024 bands when the target entropy is set to R* = 88 kbps. 

It may be noticed from Fig. |8.2(b)| that the bit budget is mainly spent on the lower 
part of the normalized MDCT spectrum. This behavior is typical for the audio signals 
we have encountered. The reason is partly that the audio signals have most of their 
energy concentrated in the low frequency region but also that the high frequency part 
is deemphasized by the perceptual weight. The perceptual weight is approximately 
proportional to the inverse of the masking curve and at the high frequency region the 
steep positive slope of the threshold in quiet dominates the masking curve. We remark 
that the bit allocation effectively limits the band width of the source signal since high 
frequency bands are simply discarded and it might therefore prove beneficial (percep- 
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Figure 8.2: Differential and discrete entropies for the normalized MDCT coefficients { expressed 
in bit/dim. ). 



tually) to use some kind of spectral band replication at the decoder in order to recover 
some of the lost high frequency components. 

The entropy Rk describes the total entropy assigned to the A:*'* band (or the fc*'' 
subset of bands if vector quantization is applied). If the number of descriptions is K 
then each side description operates at an entropy of Rk/ K bit/dim. Knowledge of Rk, 
the differential entropy h(Yk), the number of descriptions K and the packet-loss prob- 
ability p makes it possible to find the scaling factors Vk and A^fc of the central and side 
quantizers, respectively by use of ( I5.44l i and ( 15.45b . This in turn completely specify a 
MD-LVQ scheme having K descriptions. Each normalized MDCT coefficient or vec- 
tor of coefficients yk is then first quantized with the central quantizer QkiVk) — 
after which index assignments a{Xc^) — {Aq^, . . . , ^Kk-i} performed in order 
to find the codewords of the side quantizers. The codewords of the side quantizers 
are losslessly coded and put into K individual packets. Each packet then contains AI' 
encoded codewords. 

It is required that the perceptual weight fi is somehow transmitted to the decoder 
in order to be able to reconstruct. Since the K packets have an equal chance of getting 
lost we need the perceptual weight in all packets, which leads to a certain degree of 
overhead. In the case where more than one packet is received we therefore waste 
bits. It might be possible to apply some sort of MD coding on the perceptual weight 
in order to decrease the amount of side information which needs to be duplicated in 
all packets. However, it is outside the scope of this chapter to investigate the many 
aspects of perceptual lossless coding of n and we refer the readers to the work of [96] 
for more details. In the following we will simply assume that the perceptual weight 
can be perceptually lossless coded at 4 kbps, hence if the target entropy is R* — 96 
kpbs and two packets are to be used, the entropy we can actually use for MD-LVQ is 
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then only 88 kbps, since 8 kbps (4 kbps in each packet) are used for the weight. If 
a greater number of packets is desired the overhead for transmitting jj. increases even 
further. 

8.2.2 Decoder 

At the receiving side an estimate y of the normahzed MDCT spectrum is first obtained 
by simply taking the average of the received descriptions, i.e. yk = 12iei' ^'fc' 
where /' denotes the indices of the received descriptions and k' = \l'\. This estimate is 
then denormalized in order to obtain x, i.e. Xk = yk/ \/lMt- Finally the inverse MDCT 
(including overlap-add) is applied in order to obtain an approximation s of the time 
domain signal s. The decoding procedure is shown in Fig. |8.1(b)l 

8.3 Results 

In this section we compare numerical simulations with theoretical results and in addi- 
tion we show the results of a subjective listening test. We first show results related to 
the expected distortion based on the packet-loss probabilities and then we show results 
for the case of scalable coding. In both cases we assume a symmetric setup. 

8.3.1 Expected Distortion Results 

For the objective test we use four short audio clips of different genres (classical jazz 
music, German male speech, pop music, rock music) each having a duration between 
10 and 15 sec. and a sampling frequency of 48 kHz. We refer to these fragments 
as "jazz", "speech" , "pop" and "rock". We set the target entropy to 96 kbps (as 
was done in [119]) which corresponds to 2 bit/dim. since the sampling frequency is 
48 kHz. We do not encode the perceptual weight but simply assume that it can be 
transmitted to the receiver at an entropy of 4 kbps. Since the weight must be included 
in all of the K descriptions we deduct AK kbps from the total entropy, hence the 
effective target entropy R* is given by Rl = R* — AK so that a single description 
system has i?* = 92 kbps whereas a four description system has i?* — 80 kbps (i.e. 
20 kbps for each side description). For simplicity we furthermore assume that the 
sources are stationary processes so that we can measure the statistics for each vector 
of MDCT coefficients upfront. However, since audio signals in general have time 
varying statistics we expect that it will be possible to reduce the bit rate by proper 
adaptation to the source. Since for this particular test we are merely interested in 
the performance of the proposed audio coder with a varying number of descriptions 
we will not address the issue of efficient entropy coding but simply assume that the 
quantized variables can be losslessly coded arbitrarily close to their discrete entropies. 
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Table l8.2l shows the discrete entropies of the quantized normahzed MDCT coefficients 
for the four test fragments. 

K = 2 X = 2 K ^3 K ^3 K = i K = 4 

kbps bit/dim. kbps bit/dim. kbps bit/dim. 

96.22 LOO 97.09 067 96.87 051 

93.48 0.98 96.00 0.67 96.47 0.50 

93.35 0.98 95.25 0.66 95.57 0.50 

93.76 0.98 95.38 0.66 95.60 0.50 



Table 8.2: Numerical measured output entropies in kilobits per second (kbps) and hit/dim. per 
description. The target entropy is R* = 96 kbps or 2 bit/dim. 

We block the normalized MDCT spectrum into vectors of length two and use the 
Z"^ lattice vector quantizer Because of the short duration of the test fragments the 
resulting expected distortions depend upon the realizations of the packet loss patterns. 
This phenomena has been noted by other authors, cf. [3]. We therefore decided to av- 
erage the distortion results over three different loss patterns obtained by using different 
seeds to the random number generator The numerically obtained expected distortions 
are shown in Tables [QlCSl and Figs. |8.3(a)| and |8.3(b)| 



K = 1 


P = 10% 


p = 30% 


p = 50% 


jazz 


18.17(18.15) 


22.94 (23.12) 


25.16(25.23) 


speech 


17.84(17.79) 


22.61 (22.82) 


24.83 (24.86) 


pop 


17.89(17.83) 


22.66 (22.83) 


24.88 (24.91) 


rock 


18.20(18.20) 


22.97 (23.12) 


25.18 (25.23) 



Table 8.3: Theoretical (numerical) expected distortions expressed in dB for K — 1 and p = 
10, 30 and 50%. The target entropy is R* — 96 kbps or 2 bit/dim. 



K = 2 


P = 10% 


p = 30% 


p = 50% 


jazz 


9.44(10.42) 


17.96(18.33) 


22.24 (22.38) 


speech 


8.80 (9.94) 


17.55(18.04) 


21.88 (21.80) 


pop 


9.04(1032) 


17.62(18.22) 


21.94 (22.11) 


rock 


9.70(10.66) 


18.00(18.36) 


22.27 (22.39) 



Table 8.4: Theoretical (numerical) expected distortions expressed in dB for K = 2 and p = 
10, 30 and 50%. The target entropy is R* — 96 kbps or 2 bit/dim. 

As can be seen in Figs. |8.3(a)| and |8.3(b)| the expected distortions depend not only 
on the packet-loss rate but also upon the number of descriptions. At high packet-loss 
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K = i 


P = 10% 


p = 30% 


p = 50% 


jazz 


17.54(17.49) 


18.80(18.76) 


21.39 (21.34) 


speech 


15.62(15.50) 


17.34(17.29) 


20.51 (20.56) 


pop 


16.38(16.28) 


17.85(17.75) 


20.76 (20.66) 


rock 


17.44(17.33) 


18.75(18.63) 


21.38(21.29) 



Table 8.5: Theoretical (numerical) expected distortions expressed in dB for K — 3 and p = 
10, 30 and 50%. The target entropy is R* — 96 kbps or 2 bit/dim. 



K 


p = 10% 


p = 30% 


p = 50% 


jazz 


20.39 (20.35) 


20.61 (20.59) 


21.65 (21.59) 


speech 


18.88(18.75) 


19.17(19.18) 


20.52 (20.42) 


pop 


19.14(19.08) 


19.41 (19.46) 


20.70 (20.71) 


rock 


20.27 (20.18) 


20.50 (20.44) 


21.58(21.50) 



Table 8.6: Theoretical (numerical) expected distortions expressed in dB for K = 4 and p = 
10, 30 and 50%. The target entropy is R* — 96 kbps or 2 bit/dim. 



rates it is advantageous to use a higher number of packets. To verify these findings 
we performed an additional subjective comparison test. We chose three different frag- 
ments (jazz, speech and rock) and three different packet-loss rates (p = 0.1, p = 0.3 
and p — 0.5). We then performed a standard MUSHRA test [66]. At each packet-loss 
rate the original signals were encoded using K — 1,2,3 and 4 descriptions. Also 
included in each test were the hidden reference and two anchor signals (3.5 kHz and 
7 kHz lowpass filtered signals). We used nine (non-experts) listeners in the listen- 
ing test and the results are shown in Figs. IK.2l4K.4l in Appendix iKl for the individual 
fragments averaged over the nine participants. The circles in the figures denote mean 
values and the bars describe 95% confidence intervals. Fig. l8.4l shows the result when 
averaging over participants and fragments. Notice that for p = 0.3 and p = 0.5 there 
is a significant preference for using more than two descriptions. 

The results of the subjective listening tests show generally no significant differ- 
ence between the two and three packet versions for a packet-loss rate of p = 0.1, 
cf. Figs. IK.2t a)- IK.4f a). However, the results based on the perceptual distortion 
measure reveals that at p = 0.1 it is beneficial to use two packets instead of three, 
cf. Figs. |8.3(a)| and |8.3(b)| In fact, a reduction in distortion of about 7 dB can be 
achieved. This discrepancy can be partly explained by our implementation of the the 
bit-allocation strategy outlined in Section [8.1.5l To avoid assigning a too small rate 
to a given frequency band (which then would violate the high-resolution assumptions) 
we have, in the experiments described above, excluded MDCT bands which were as- 
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10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 

Packet-loss probability [%] Paokt^t-loss probability \%] 



(a) jazz (b) speech 

Figure 8.3: The expected distortion as a function of packet-loss probabilities for MD-LVQ when 
operating at a target entropy of 96 khps. 



signed a rate lower than 3 bit/dim. per description|fj The effect of this is that the high- 
resolution approximations are good so that theoretical and numerical results agree but 
the downside is that the input signal is severely lowpass filtered. The contribution of 
the high frequency bands to the total distortion is therefore high, hence, the recep- 
tion of more than two descriptions does not improve the quality of the reconstructed 
signal much. In addition we would like to emphasize two important factors which 
might also contribute to the inconsistency between subjective listening tests and the 
perceptual distortion measure. First of all, the perceptual distortion measure is based 
upon a single block at a time and therefore the continuity of the signal over time is not 
addressed]^ Secondly, the distortion measure is defined in the MDCT domain and 
since the MDCT is not an orthogonal transform the distortion in the MDCT domain is 
not equivalent to the distortion in the time domain. 

As previously mentioned we have in these tests excluded MDCT bands where 
the rate assignment is less than 3 bit/dim. per description to make sure that the high- 
resolution assumptions are valid. Such an approach excludes a great amount of MDCT 
bands (especially those representing the high frequency contents of the signal) and the 
coded signal sounds muffled (lowpass filtered). The reasoning behind this choice is 

*'lf the numerically measured discrete entropy is, for example, 0.1 bit/dim. greater than the specified 
theoretical entropy, then, since the sampling frequency is 48 kHz, the resulting bit rate is 4.8 kbps above the 
target entropy. Furthermore, if this 0. 1 bit/dim. gap is per description, then, in a three-description setup, the 
resulting rate would exceed the target rate by 14.4 kbps. Practical experiments have shown that at 3 bit/dim. 
per description, the numerically measured discrete entropy is off by less than 0.03 bit/dim. per description 
for a range of index values. 

*^The listeners agreed that the "hick-ups" resulting from time gaps due to packet losses were the most 
annoying artifacts present in the coded signals. The overlapping nature of the MDCT is, however, able to 
reduce the impact of isolated packet losses. 
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p = 10% 



p = 30% 



p = 50% 
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90 
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Figure 8.4: MUSHRA test results averaged over all three audio clips for p = 0.1, 0.3 and 
p = 0.5. The seven signals appear in the following order: Hidden ref, 3.5 kHz, 7 kHz, K = 
1,K = 2,K = 3andK = 4. 



that a "lowpass" filtered version of the signal (without time gaps) is often preferable 
over a full bandwidth signal (with time gaps). Alternatively, we may take into ac- 
count that the practical rate becomes too high for the bands that are assigned a too low 
theoretical rate. Thus, we can heuristically assign a lower target rate for the MDCT co- 
efficients representing the higher frequency bands. Since we encode two-dimensional 
vectors there are 512 bands in total but only about the first 300 of these are assigned 
a positive rate. We then modify the scale factor Vk for the k*^ band by the following 
rule 



1.0 -Vk, 
1.3 • Uk, 

1-5 • Vk, 



< fc < 50, 
51 < < 100, 
101 < /c < 200, 
201 < A: < 250, 
251< A: < 300. 



(8.21) 



For each different fragment we set the target R* such that the practical rate is very 
close to 96 kbit/sec. (incl. 4 kbit/sec. per packet for the masking curve). These rates 
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Fragment R* [kbit/sec] H{Y) [kbit/sec] 



jazz 


92.16 


95.92 


harpsi 


86.40 


96.34 


speech 


91.92 


96.03 


pop 


89.76 


96.29 


rock 


94.08 


96.09 



Table 8.7: The target rate R* is set lower than 96 kbit/sec. which leads to a practical rate 
H{Y) close to 96 kbit/sec. 



are shown in Table 18. 7 Pl 

The numerically measured expected distortions based on the packet-loss probabil- 
ities are shown in Fig. |8.5(a)| for the jazz fragment. We have swept the packet-loss 
probability between 1% and 50% in steps of 1%. Each test is repeated 10 times to 
reduce the influence of a particular loss pattern. Notice that already at packet-loss 
probabilities as low as one percent it becomes advantageous to use three descriptions 
instead of two descriptions. Fig. |8.5(b)| shows the results of a similar experiment for 
the speech fragment. 




Packet-loss probability [%] 

(a) jazz fragment 



Packet-loss probability [%] 

(b) speech fragment 



Figure 8.5: Expected distortion as a function of packet-loss probabilities. 



8.3.2 Scalable Coding Results 

We now assess the improvement of audio quality as more packets are received. This 
is a form of scalable coding, where some receivers have access to more information 

^'in this experiment we have included an additional audio fragment "harpsi". which consists of "music" 
from a Harpsichord. 
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Impairment ITU-R Grade ODG 



Imperceptible 


5.0 


0.0 


Perceptiple, but not annoying 


4.0 


-1.0 


Slightly annoying 


3.0 


-2.0 


Annoying 


2.0 


-3.0 


Very annoying 


1.0 


-4.0 



Table 8.8: Relationship between the ITU-R 5-grade impairment scale and the ODGs [12]. 



Fragment 


(Ao) 


(Ai) 


(A2) 


Avg. 


jazz 


-2.652 


-2.571 


-2.720 


-2.647 


harpsi 


-1.976 


-1.757 


-2.606 


-2.113 


speech 


-2.649 


-2.492 


-2.961 


-2.701 


pop 


-3.328 


-3.375 


-3.445 


-3.383 


rock 


-2.699 


-2.556 


-2.787 


-2.681 



Table 8.9: ODGs when receiving a single description out of three. 



(descriptions) than others. In this case no description losses occur Instead of using the 
expected distortion we will use the Objective Difference Grade (ODG) based on the 
Matlab implementation by Kabal et al. [70] of the PEAQ standard [108]. The ODGs 
are related to the standard ITU-R 5-grade impairment scale as shown in Table 18.81 
Tables [0]48 .111 show the ODGs for the five different test fragments. The last column 
of Tables [8791 and 18 . 1 Ol show the mean ODGs when averaged over the three different 
combinations of descriptions. These average ODGs as well as the results of Table lS.llI 
are also shown in the bar diagram in Fig. 18.61 

From the tables it may be observed that the perceptual distortion is approximately 
symmetric, i.e. the ODG is essentially independent of which packet is received. In 
addition, it can be seen that as more packets are received a substantial improvement in 
quaUty can be expected. 



Fragment 


(Ao,Ai) 


(Ao,A2) 


(Ai,A2) 


Avg. 


jazz 


-1.033 


-1.162 


-1.021 


-1.072 


harpsi 


-0.729 


-0.993 


-0.893 


-0.872 


speech 


-0.994 


-1.171 


-1.040 


-1.068 


pop 


-1.897 


-2.401 


-2.082 


-2.127 


rock 


-1.125 


-1.284 


-1.128 


-1.179 



Table 8.10: ODGs when receiving two descriptions out of three. 
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Fragment (Ao,Ai,A2) 



jazz 


-0.104 


harp si 


-0.166 


speech 


-0.189 


pop 


-0.171 


rock 


-0.184 



Table 8.11: ODGs when receiving all three descriptions. 



IT 



^ -1 

6 

9, -1-5 



O -2.5 



-3.5 



jazz 



harpsi 



speech 



pop 



I One desc. 
I Two desc. 
] Three desc. 

rock 



Figure 8.6: ODGs for the reception of one to three packets out of three for different test frag- 
ments. 



8.4 Conclusion 

We combined MD-LVQ with transform coding in order to obtain a perceptually ro- 
bust audio coder. Previous approaches to this problem were restricted to the case of 
only two descriptions. In this work we used ii'-Channel MD-LVQ, which allowed for 
the possibility of using more than two descriptions. For a given packet-loss proba- 
bility we found the number of descriptions and the bit allocation between transform 
coefficients, which minimizes a perceptual distortion measure subject to an entropy 
constraint. The optimal MD lattice vector quantizers were presented in closed form, 
thus avoiding any iterative quantizer design procedures. The theoretical results were 
verified with numerical computer simulations using audio signals and it was shown 
that in environments with excessive packet losses it is advantageous to use more than 
two descriptions. We verified in subjective listening tests that using more than two 
descriptions lead to signals of perceptually higher quaUty. 
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9.1 Summary of Results 

We presented an index-assignment based design of iiT-channel MD-LVQ. Where 
previous designs have been Umited to two descriptions we considered the general 
case of K descriptions. Exact rate-distortion resuhs were derived for the case of 
K < Z descriptions and high resolution conditions for smooth stationary sources 
and the squared error distortion measure. In the asymptotic case of large lattice vector 
quantizer dimension and high resolution conditions, it was shown that existing rate- 
distortion MD bounds can be achieved in the quadratic Gaussian case. These results 
were conjectured to hold also for K > i descriptions. 

In the two-description asymmetric case it was shown that the performance was su- 
perior to existing state-of-the-art asymmetric schemes in finite lattice vector quantizer 
dimensions greater than one. In one and infinite dimensions as well as in the symmet- 
ric case (for all dimensions), the performance is identical to existing state-of-the-art 
schemes. 

In the three-description synnmetric and asymmetric cases for finite lattice vector 
quantizer dimensions, the rate loss of the proposed design is superior to that of existing 
schemes. 

The optimal amount of redundancy in the system was shown to be independent 
of the source distribution, target rate and type of lattices used for the side quantizers. 
Basically, the channel conditions (expressed through a set of packet-loss probabilities) 
describe the required amount of redundancy in the system. Thus, for given channel 
conditions, the optimal index-assignment map can be found and adapting to time- 
varying source distributions or bit rate requirements amounts to a simple scaling of 
the central and side lattice vector quantizers. 

We proposed an entropy-constrained design where either the side description rates 
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or their sum rate are subject to entropy contraints. In the case of a single sum rate 
entropy contraint, we showed that the optimal bit allocation across descriptions is not 
unique, but in fact consists of a set of solutions, which all lead to minimal expected 
distortion. 

On the practical side it was shown that the optimal X-channel MD lattice vector 
quantizers can be found in closed-form, hence avoiding any iterative (e.g. generalized 
Lloyd-like) design algorithms. Furthermore, we combined MD-LVQ with transform 
coding in order to obtain a perceptually robust audio coder. Previous approaches to 
this problem were restricted to the case of only two descriptions. For a given packet- 
loss probability we found the number of descriptions and the bit allocation between 
transform coefficients, which minimizes a perceptual distortion measure subject to 
an entropy constraint. The theoretical results were verified with numerical computer 
simulations using audio signals and it was shown that in environments with excessive 
packet losses it is advantageous to use more than two descriptions. We verified in 
subjective listening tests that using more than two descriptions leads to signals of 
perceptually higher quahty. 

9.2 Future Research Directions 

In this thesis we considered index-assignment based MD schemes at high resolution 
conditions, which provide a partial solution to the _fC-channel MD problem. However, 
more work is needed before the general MD problem is solved. Besides the informa- 
tion theoretic open problems discussed in Chapter|4]there are many unsolved problems 
related to MD-LVQ. Below we list a few of these. 

• Proving the conjectures of this thesis, i.e. proving the rate-distortion results for 
K > i descriptions. 

• Extending the results to general resolution. To the best of the authors knowl- 
edge, the only case where exact rate-distortion expressions (in non high- 
resolution cases) have been presented for index-assignment based MD schemes, 
is the two-channel scalar scheme by Frank-Dayan and Zamir [39]. 

• It is an open problem of how to construct practical MD-LVQ schemes that 
comes arbitrarily close to the known MD bounds. Such schemes require high- 
dimensional lattice vector quantizers and large index values. However, solving 
the linear assignment problem can become computationally infeasible for large 
index values. For the symmetric case and certain low dimensional lattices, some 
progress have been made in reducing this complexity by the design of Huang 
and Wu [65]. A construction for high-dimensional nested lattice codes was re- 
cently presented by Zamir et al. [162]. No index-assignment methods have, 
however, been presented for the nested lattice code design and the problem is 
therefore not solved. 
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• Constructing functional MD schemes for existing applications in real environ- 
ments and assessing their performance. For example for real-time application, 
even if the packet-loss rate of a network is very low, the delay might occa- 
sionally be high, which then means that (at least for real-time applications) a 
delayed packet is considered lost (at least for the current frame) and the use of 
MD coding might become beneficial. 
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We will here briefly define the Quaternions and describe a few important properties 
that we will use in this work. For a comprehensive treatment of the Quaternions we 
refer the reader to [23,71, 150]. 

The Quaternions, which were discovered in the middle of the 19*'* century by 
Hamilton [71], is in some sense a generalization of the complex numbers. Just as 1 
and i denote unit vectors of the complex space C we define 1, i, j and k to be unit 
vectors in Quaternion space H. The set of numbers defined as {a + 6i + cj + dfc : 
a, b,c,d E M} are then called Quaternion numbers or simply Quaternions. Addition 
of two Quaternions q = a + bi + cj + dk and q' = a' + b'i + c' j + d'k is defined as 

q + q' = {a + a') + {b + b')i + (c + c')j + {d + d')k, (A.l) 

and multiplication follows by first defining a multiplication rule for pairs of Quater- 
nion units, that is 







= fc2 = 


-1 


ij 


^k, 


ji = 


-fc, 


jk 


= h 


kj = 


-1, 


ki 


= h 


ik = 


-j, 



(A.2) 



which leads to 

qq' = (aa' - bb' - cd - dd') + iab' + ba' + cd! - dc')i 

(A. 3) 

+ {ad + ca' + db' - bd')j + {ad! + da' + bd - cb')k. 

Definition A.1.1. The skew field H of Quaternions is defined as the set {z — 

a + bi + cj + dk : a,b,c,d E M.} combined with two maps (addition and multi- 
plication) given by dA.lt and ( |A.3l l. respectively, that satisfies field properties except 
that multiplication is non commutative. 
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Definition A.1.2. Quaternionic conjugation of q = a -\- bi + cj + dk € M is given 
by q* ~ a — bj — cj — dk and we denote by Quaternionic conjugation and vector 
transposition where q e . 

Definition A.1.3. The real part of a Quaternion g — a+bi+cj+dk is gi\enhydi{q) = 
a and the complex part (also called the vector part) is given by ^{q) = bi + cj + dk. 

Lemma A.1.1 ( [6]). Two Quaternions q^ and qi commute, i.e. go^i = QiQo if their 
vector parts are proportional (i.e. linear dependent) or, in other words, if the cross 
product 3 (go) x ^^(91) = 0. 



Lemma A.1.2 ( [22]). The norm \\q\\ of a Quaternion q £ 
and satisfies the usual vector norm, i.e. 



:is given by \\q\\ 



/q*q 



62 



d2. 



Definition A.1.4 ( [14]). The Quaternions can be represented in terms of matrices. 
The isomorphic map 0^ : (H, +, •) — > {H4X4, ®, ®) between the space H of Quater- 
nions and the space H^xa of 4 x 4 matrices over the real numbers R defined by 



0^(0 + bi + cj + dk) 



a 


-b 


— c 


-d- 


b 


a 


-d 


c 


c 


d 


a 


-6 


d 


—c 


b 


a 



(A.4) 



describes left multiplication by the Quaternion q. Similar we define right multiplica- 
tion by the map : (H, +, •) (-^4x4, ©, <Xi) given by 



0fl(a + bi + cj + dk) 



-b 
a 

-d 
c 



-c 

d — c 

a b 

-b a 



(A.5) 



It follows from Definition IA.1.41 that addition and multiplication of two Quater- 
nions can be done by use of the usual matrix addition and matrix multiplication (g). 
Furthermore, Quaternionic conjugation can easily be done in H4X4 space where it is 
simply the matrix transpose. 
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In this appendix we give a brief introduction to the theory of algebraic modules. For 
more information we refer the reader to the following textbooks [1, 22, 61, 72]. 

B.l General Definitions 

Definition B.1.1. Let ^ be a ring which is not necessarily commutative with respect 
to multiplication. Then an Abelian (commutative) group 5* is called a left ^-module 
or a left module over ^ with respect to a mapping (scalar multiplication on the left 
which is simply denoted by juxtaposition) ^ x 5 — > 5 such that for all a, 6 e ^ 
and g,h G S, 

1) a{g + h) = ag + ah, 

2) (a + b)g = ag + bg, 

3) {ab)g = a{bg). 

Remark B.1.1. For simplicity we have used the same notations for addi- 
tion/multiplication in the group as well as in the ring. 

Remark B.l. 2. A right module is defined in a similar way but with multiplication on 
the right. In fact if the ring ^ is commutative then every left ^-module is also a 
right ^-module [61]. 

Definition B.1.2. If ^ has identity 1 and if la = a for all a G S, then S is called a 
unitary or unital ^-module. 

Remark B.l. 3. In this work all modules have an identity. 
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Definition B.1.3 ( [72]). A subset 5" = {mi, . . . ,m„} C S' of the ^-module S 
is linearly independent over ^ if, for e ^ , xinii + ■ ■ ■ + x„m„ = only if 
xi = • • • = = 0. If in addition S' generates S then 5' is a basis for S. 

Example B.1.1. The set S = {2,3} is finite and generates Z, considered as a Z- 
module over itself [72]. However, 5 is not a linearly independent set. Further, neither 
element of S can be omitted to give a generating set with one member. Hence, S is 
not a basis of Z. 

Definition B.1.4 ( [72]). A ^-module that has a basis is called a free ^-module. 

Definition B.1.5 ( [72]). The number of elements in a basis of a ^-module S is 
called the rank or dimension of S. 

Remark B.1.4. Not all modules have a basis [72]. Let Z^ = Z/mZ be the residue 
ring of integers, i.e. Z^ = {[0], . . . , [to — 1]}, where [s] = [r] in Z-^ implies s = r 
(mod to). Notice that Z^ contains no Unear independent subsets, since mx = for 
any a; G Z, hence Z^ has no basis and is therefore not a free module. 

Definition B.1.6 ( [1]). Let ^ be a ring and let S, S' be left ^-modules. A function 
f : S ^ S' is a ^-module homomorphism if 

1. /(toi + TO2) = f{mi) + /(TO2) for all toi, TO2 G S', and 

2. f{am) = af{m) for all a e ^ and m G S. 

Definition B.1.7 ( [1]). The set of all ^-module homomorphisms from S to S' is 

denoted Hom(S', S'). If S = S' then we write End(S') where elements of End(S') are 
called endomorphisms. If / G End(S') is invertible, then it is called an automorphism. 
The group of all automorphisms is denoted Aut(S'). 

Definition B.1.8 ( [72]). Let ^ be a ring and S a ^-module. Then an annihilator of 
an element gf e 5 is the set 

Ann(3) = {h € J : hg = 0}. (B.l) 

An element g G S is said to be a torsion element of S if Aim{g) ^ 0, that is, there is 
some non-zero element a G ^ with ag = 0. 

Definition B.1.9 ( [72]). A ^-module S is said to be torsion-free if the only torsion 
element in 5 is 0. 

Definition B.l. 10. Let 5" be a finite group and let 5* be a left ^-module. The orbit 
under the action of to € S is obtained by left multiplication, i.e. S'm = {gm : g G 
S'}. 
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B.2 Submodule Related Definitions 

Definition B.2.1. Let 5 be a ^-module and S' a nonempty subset of S. Then S' is 
called a submodule of 5 if S" is a subgroup of S and for all g G S,h G S', we have 

gh e S". 

Definition B.2.2 ( [61]). A cyclic submodule is a submodule which is generated by 
a single element. For example in a left ^-module S, a cyclic submodule can be 
generated by m e 5 in the following ways ^m = {xm : x G ^} orm^ = {mx : 
XG^}. 

Definition B.2.3 ( [61]). In general the submodule S' of the ^-module S generated 
by a finite subset {mi , . . . , m„} C 5 is the set 

J^mi H h ^run = {ximi H l-a;„m„ :xi,...,XiG J} (B.2) 

of all Unear combinations of mi, ... , m„. Such submodules are called finitely gener- 
ated. MS' = S then {mi, . . . , m„} is a set of generators for S. 

Lemma B.2.1 ( [72]). Let 5 and 5' be submodules of a ^-module S. Then their 
sum 

S^S' = {l + n:lGS,nGS'}, (B.3) 
is also a submodule. Moreover, S + S' = S S' C S. 

Lemma B.2.2 ( [72]). Let S and S' be submodules of a ^-module S. Then their 
intersection 

SnS' = {x : X e S undx e S'}, (B.4) 

is also a submodule. Moreover, S n S' = S <;=^ S' C S". 

Remark B.2.1. Submodules of a vector space are its subspaces. 

Proposition B.2.1 ( [72]). Let S' be a submodule of the ^-module S. Let m, n e S 
and define a relation on S by the rule that m = n <J=^ m~ n G S'. The equivalence 
class of an element m e 5 is given by the set 

[m]=m + S' = {m + l:lGS'}. (B.5) 

The quotient module (or factor module) S/ S' is defined to be the set of all such equiv- 
alence classes, with addition given by 

[m] + [n] = [m + n], [m], [n] G S/S', (B.6) 

and multiplication by r e ^ is given by 

r[m] = [rm],[m]G S/S'. (B.7) 
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Definition B.2.4 ( [72]). The module homomorphism n : S S/S' defined by 
7T{m) = [m] is called the natural map (or canonical homomorphism) from S to S/ S'. 

Definition B.2.5. Let S" and S" be arbitrary submodules of the ^-module S. If 
S' + S" S and S' O S" ^ their configuration or relationship is often expressed 
in the diagram shown in Fig. |B.l(a)| If S is the direct sum of S' and S" we have 
S" n S"' = and S" + S" = S which lead to the simpler diagram shown in Fig. |B.l(b)| 




B.3 Quadratic Forms 

Definition B.3.1 ( [1]). A conjugation on ^ is a function c: ^ ^ ^ satisfying 

1. c(c(0)=e, Vee^ 

2. c(a+6)-c(a) + c(6), va,6e^ 

3. c(ei6) = c(Ci)c(6), Vei,6G^ 

Definition B.3.2 ( [1]). Let S* be a free ^-module. A bilinear form on 5* is a function 

(j) S X S —f ^ satisfying 

1. <^(Cia;i +i2X2-,y) = Ci0(2;i,2/) +60(2^2, y) 
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2. (I){x, Ciyi + 62/2) = yi) + ^2(f>{x, y2) 
forall a;i,a;2,yi,y2 € S and ^1,^2 € 

Definition B.3.3 ( [1]). Let 5 be a free ^-module. A sesquilinear form on is a 
function (p : S x S ^ satisfying 

1. (\){ixxx +£,2X2,y) = ^i<j){xi,y)+£,2(l){x2,y) 

2. </)(a;,Ciyi +62/2) = c{^i)(l){x,yi) + c{^2)<P{x,y2) 

for all xi,X2,yi,y2 G 5 and ^1, 1^2 G ^ for a non-trivial conjugation ^ 1-^ c(^) on 
J- 

Definition B.3.4 ( [22]). Let A be a ^-lattice in M-^ having basis vectors Ci , ■ • ■ , Cl € 
whose transposes form the rows of the generator matrix M. Then any lattice point 
A e A may be written on generic form as A = ^ M where <^ — (^1, . . . , ^i)-^ and 
where & J' . Let us define the following function of A (i.e. a squared norm) 

1=1 j=i (^■'^> 

The function t in ( IB.8I ) is referred to as the quadratic form associated with the lat- 
tice A. If A has full rank, then MM^ is a positive definite matrix and the associ- 
ated quadratic form is called a positive definite form. If we extend r to t(Ai, A2) = 
^^MM^^2 we obtain the bilinear form of Definition |B.3.2| and if the underlying field 
is non real we get the sesquiUnear form of Definition IB. 3. 31 where the conjugation 
function c depends on the field. 

Remark B.3.L In this work we will not make explicitely use of quadratic forms. In- 
stead we equip the underlying field with an inner product (•, •) which satisfies Defi- 
nition |B]32] and is therefore a bilinear form. 

Example B.3.1. Let A C M^, i.e. A is a lattice embedded in the field M^. Then we 
can define the usual vector norm ||A|p = (A, A), where A G A. Notice that here the 
conjugation is simply the identity. See Appendix IClfor more examples. 
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In this appendix we present a number of lattice-related definitions and properties 
which are used throughout the thesis. 

Let V be a vector space over the field K and let V be equipped with an inner 
product (•,•). If — then V is the traditional vector space over the Cartesian 
product of the reals@ As such, (V, (•, •)) is an inner-product space. An inner-product 
space induces a norm || • || defined as || • |p = (•, •). If — we use the £2- 
norm defined as \\x\\^ ^ x^x whereas if = we have ||x||^ = x^x where 
^ denotes Hermitian transposition (i.e. the conjugate transpose). If — then 
= x^x where ^ denotes Quaternionic conjugation and transposition. For more 
information about inner-product spaces we refer the reader to the widely used textbook 
by Luenberger [89]. 

C.l General Definitions 

Definition C.1.1 ( [22]). A lattice A C consists of all possible integral linear 
combinations of a set of basis vectors, or, more formally 



where Q £ are the basis vectors also known as generator vectors of the lattice and 
^ C K is a well defined ring of integers. 

Remark C.1.1. It should be noted that it is often convenient to use L' > L basis 
vectors to form an L-dimensional lattice embedded in . 

**Recall that, in a vector space over M^, addition and subtraction is witli respect to vectors of wliereas 
multiplication is defined as multiplications of vectors in with scalar elements of R. Thus, a vector space 
is a ^-module where ^ is a field, i.e. a ring where all elements (except 0) have inverses. 




(C.l) 
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Definition C.1.2. Let M be a generator matrix of the lattice A. Then the rows of M 
are given by the tranposes of the column vectors ,i = 1, . . . ,L, where we actually 
do not require that M is square. 

Definition C.1.3. The square matrix A = MM^ is called the Gram matrix. 

Definition C.1.4. A fundamental region of a lattice is a closed region which contains 
a single lattice points and tessellate the underlying space. 

Lemma C.1.1 ( [22]). All fundamental regions have the same volume. 

Lemma C.1.2 ( [22]). The fundamental volume of A is given by = ^det(A), 
sometimes written as = det(A). If M is a square generator matrix then u = 

|det(M)|. 

Definition C.1.5 ( [25]). An i-dimensional polytope is a finite convex region in 
enclosed by a finite number of hyperplanes. 

Definition C.1.6 ( [30]). The quotient K^/A is the L-dimensional torus obtained by 
combining opposite faces of the fundamental parallelotope {aid + • • • + aiCijO < 
ai < 1}. 

Definition C.1.7. The Cartesian product (g) of two lattices Ai and A2 is obtained by 
pairing all points in Ai with every point in A2, i.e. 



It follows that the dimension of A = Ai (g) A2 is equal to the sum of the dimensions 
of the two lattices Ai and A2. 

Definition C.1.8 ( [22]). The automorphism group Aut(A) of a lattice A is the set of 
distance-preserving transformations (or isometries) of the space that fix the origin and 
takes the lattice to itself. 

Theorem C.1.1 ( [22]). For a lattice in ordinary Euclidean space M^, Aut(A) is finite 
and the transformations in Aut(A) may be represented by orthogonal matrices. Let A 
have generator matrix M. Then an orthogonal matrix B is in Aut(A) if and only if 
there is an integral matrix U with determinant ±1 such that 



This implies U = MBM^A ^, where ^ ^ is the Gram matrix of A. 

Remark C. 1 .2. Aut(A = Z^) consists of all sign changes of the L coordinates (= 2^) 
and all permutations (= Ll). Hence, |Aut(Z^)| = 2^ LI [22]. 



Ai (g) A2 = {(Ai, A2)|Ai e Ai, A2 G A2}. 



(C.2) 



UM = MB. 



(C.3) 
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Definition C.1.9 ( [22]). The dual lattice A of the lattice A is given by 

A = {xe R^lx'^X e Z for all A e A}. (C.4) 

Alternatively, if M is a square generator matrix of A then A can be constructed by use 
of the generator matrix M = (M~^)^. 

Theorem C.1.2 ( [30]). If A c is a discrete subgroup with compact quotient 

R-^/A then A is a lattice. 

Definition C.1.10 ( [22]). The coefficients B., of the Theta series OaI^) = E,: 
of a lattice A describe the number of points at squared distance i from an arbitrary 
point in space (which is usually taken to be the origin). The indeterminate q is some- 
times set toq = exp{i'7rz), where z gC and Q{z) > 0. 

C.2 Norm Related Definitions 

Definition C.2.1 ( [22]). Let A c be a lattice. The nearest neighbor region of 
A e A is defined as 

V{X) = {xeK^ : \\x - Xf < \\x - X'f, VA' e A}. (C.5) 

Definition C.2.2. The nearest neighbor regions of a lattice are also called Voronoi 
cells, Voronoi regions or Dirichlet regions. In this work we will use the name Voronoi 
cells. 

Definition C.2.3. Voronoi cells of a lattice are congruent polytopes, hence they are 
similar in size and shape and may be seen as translated versions of a fundamental 
region, e.g. Vq = V{0), i.e. the Voronoi cell around the origin. 

Definition C.2.4 ( [22]). The dimensionless normalized second moment of inertia 
G{A) of a lattice A is defined by 

G(A) ^ j^^ M'dx. (C.6) 

Remark C.2.1. Applying any scaling or orthogonal transform, e.g. rotation or reflec- 
tion on A will not change G{K), which makes it a good figure of merit when compar- 
ing different lattices (quantizers). In other words, G(A) depends only upon the shape 
of the fundamental region, and in general, the more sphere-like shape, the smaller 
normaUzed second-moment [22]. 

Definition C.2.5 ( [38]). The minimum squared distance (i^;^(A) between lattice 
points is the minimum non-zero norm of any lattice point A G A, i.e. 

rfL(A) = min||Af. (C.7) 

AeA 
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Definition C.2.6 ( [38]). The packing radius Pp{A) of the lattice A C IR-^ is the radius 
of the greatest L-dimensional sphere that can be inscribed within Vq. We then have 

Pp(A) ^d™„(A)/2. (C.8) 

Definition C.2.7 ( [38]). The covering radius pc(A) of the lattice A C M-^ is the radius 
of the least L-dimensional sphere that contains Vq, i.e. 

Pc(A) = max||a;||. (C.9) 

xev„ 

Definition C.2.8 ( [38]). The kissing®] number M.{A) is the number of nearest neigh- 
bors to any lattice point, which is also equal to the number of lattice points of squared 
normd^i„(A),i.e. 

i^(A)^|{AeA:|lA|p = rfL(A)}|. (C.IO) 

Definition C.2.9. The space-filling loss of a lattice A with dimensionless normalized 
second moment G(A) is given by 

i^Loss = 10 logio (2^eG(A)) dB. (C. 1 1) 



C.3 Sublattice Related Definitions 

Definition C.3.1. A sublattice A' C A is a subset of the elements of A that is itself a 
lattice. 

Definition C.3.2 ( [28]). A sublattice A' C A is called clean if no point of A lies on 
the boundary of the Voronoi cells of A'. 

Definition C.3.3. If A' is a sublattice of A then N = |A/A'| denotes the index or 
order of the quotient A/ A'. 

Definition C.3.4. If A' is a clean sublattice of A then the index value N = |A/A'| is 
called an admissible index value. 

Definition C.3.5. The L*'* root of the index N is called the nesting ratio N', i.e. 

N' = Afi/i. 

Definition C.3.6 ( [28]). Let A be an L dimensional lattice with square generator 
matrix M. A sublattice A' C A is geometrically strictly similar to A if and only if the 
following holds 

1 . There is an invertible L x L matrix U i with integer entries 

®The terminology kissing number was introduced by N. J. A. Sloane who drew an analogy to billiards, 
where two balls are said to kiss if they touch each other, see for example the interview with N. J. A. Sloane 
by R. Calderbank, which can be found online at httpV/www.research.att.com/" njas/doc/interview.html. 
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2. a non-zero scalar Ci G R 

3. an orthogonal L x L matrix Ki with determinant 1, 
such that a generator matrix AIi for Ai can be written as 

Ml = UiM = ciMKi. 
If ( IC.I2I 1 holds then the index A^i of A' is equal to 



iVi = |A/A'| = 



dct(A') 



det(Mi) 



det(M) 



det([/i 



det(A) 

Furthermore, A' has Gram matrix 

Ai = A/iMf = UiMM^U'[ = UiAU^ = c\A, 
where A ~ MM^ is a Gram matrix for A. 



'1 • 



(C.12) 



(C.13) 



(C.14) 



Definition C.3.7. If in Definition |C. 3 .61 the determinant of Ki is allowed to be ±1, 
i.e. Ki can be either a rotation or a reflection operator, then the sublattice A' is said to 
be geometrically similar to A. 



Appendix 




Root Lattices 

This appendix describes some properties of the roo0 lattices considered in this thesis. 

D.l 

The scalar uniform lattice also called partitions the real line into intervals of equal 
lengths. Table lD. ll outlines important constants related to the lattice. 



Description 


Notation 


Value 


Dimension 


L 


1 


Fundamental volume 


V 


1 


Packing radius 


Pp 


1/2 


Covering radius 


Pc 


1/2 


Space-filling loss 




1.5329 dB 


Space-filling gain over Z^ 


^gain 


OdB 


Kissing-number 


R 


2 


Minimal squared distance 


"mill 


1 


Dimensionless normalized 


G(A) 


1/12 


second moment 



Table D.l: Relevant constants for the Z lattice. 



The set of admissible index values for Z^ is the set of all odd integers [28] and the 
coefficients of the Theta series are given by i?o = 1 and Bi = 2,i > 0. 

'"The term root lattice refers to a lattice which can be generated by the roots of specific reflection 
groups [22, 34] . 
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D.2 Z2 

A generator matrix for (also known as the square lattice) is given by 

M=[; «]. (D.i) 

The Gram matrix A is identical to the generator matrix, i.e. A = M. Table lD.2l gives 
an overview of important constants related to the lattice. 



Description 


Notation 


Value 


Dimension 


L 


2 


Fundamental volume 


V 


1 


Packing radius 


Pp 


1/2 


Covering radius 


Pc 


PpV^ 


Space-filling loss 


^loss 


1.5329 dB 


Space-filling gain over Z^ 




OdB 


Kissing-number 




4 


Minimal squared distance 


mm 


1 


Dimensionless normalized 


G(A) 


1/12 


second moment 



Table D.2: Relevant constants for the lattice. 



The first 50 coefficients of the Theta series, i.e. the number of points in each of 
the 50 first shells of A are shown in Table ID. 3] and the first seven shells are shown in 
Fig.imi 



1 ,4,4,0,4,8,0,0,4,4,8,0,0,8,0,0,4,8,4,0,8,0,0,0,0, 12, 
8,0,0,8,0,0,4,0,8,0,4,8,0,0,8,8,0,0,0,8,0,0,0,4 



Table D.3: The first 50 coefficients of the Theta series with starting point at zero for the Z^ 
lattice. 

Let A be the square lattice represented in the scalar complex domain, i.e. A = 
Then a sublattice A' = ^A, where ^ = a + ih and ^ e ^1' is clean if and only if 
N — a? + b"^ \s odd [28]. Equivalently an integer N is an admissible index value 
if it can be written as a product of primes congruent to 1 (mod 4) and/or a product 
of primes congruent to 3 (mod 4) [21,28]. This set is given by integer sequence 
A057653 [125], see also TableEl 

A subgroup r4 C Aut(A = Z^) of order 4 is given by ( |2.9l l. 
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-3-2-1 1 2 3 



Figure D.l: The first 7 non-zero shells of is here shown as large circles (incl. the one at 
the origin ). Notice that the number of points lying on each circle agrees with the corresponding 
coefficient of its Theta series. 

D.3 A2 

The hexagonal lattice (also known as A2) can be represented in the complex field 
where it is identical to S'. When represented in K.^ a possible generator matrix is 

M 

Its Gram matrix is given by 

A 

Table ID .5] summarizes important constants related to the A2 lattice. 

Let A be the hexagonal lattice represented in the scalar complex domain, i.e. A = 
(f. Then a sublattice A' = ^A, where — a + ojb and ^ S is clean if and only 
if a and h are relative prime or equivalently if and only if TV is a product of primes 
congruent to 1 (mod 6) [28]. This set is given by integer sequence A004611 [125], 
see also Table lD.6] 

A subgroup Fe C Aut(A = S) of order 6 is given by the rotational group 

r6 = {exp(ifc7r/6),A: = 0, ...,5}. (D.4) 



1 
A/2 V3/2 



1 -1/2' 
-1/2 1 



(D.2) 



(D.3) 
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1,5,9,13,17,25,29,37,41,45,49,53,61,65,73,81,85,89,97,101,109,113, 
117,121,125,137,145,149,153,157,169,173,181,185,197,205,221,225, 
229,233,241,245,257,261,265,269,277,281,289,293,305,313,317,325 
333,337,... 

Table D.4: Admissible index values for . 



Description 


Notation 


Value 


Dimension 


L 


2 


Fundamental volume 


V 


\/3/2 


Packing radius 


Pp 


1/2 


Covering radius 


Pc 




Space-filling loss 


-^loss 


1.3658 dB 


Space-filling gain over 




0.1671 dB 


Kissing-number 




6 


Minimal squared distance 


mm 


1 


Dimensionless normalized 
second moment 


G(A) 


5/(36V3) 



Table D.5: Relevant constants for the A2 lattice. 



The first 50 coefficients of the Theta series for A2 are shown in Table ID. 71 and 
Fig. EH 



D.4 

The hypercubic lattice is generated by 



M 



"1 








0" 





1 














1 





.0 








1 



(D.5) 



The Gram matrix A is identical to the generator matrix, i.e. A = M. Table ID. 81 gives 
an overview of important constants related to the lattice. 

has a geometrically-similar and clean sublattice of index N if and only if N is 
odd and of the form for some integer a [28]. The set of admissible index values is 
given by integer sequence A016754 [125], see also Table ID. 9l 

The first 50 coefficients of the Theta series for Z^ are given in Table ID. lOl 
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1,7,13,19,31,37,43,49,61,67,73,79,91,97,103,109,127,133,139, 
151,157,163,169,181,193,199,211,217,223,229,241,247,259,271, 
277,283,301,307,313,331,337,343,349,361,367,373,379,397,403, 
409,421,427,433,439,457,. . . 



Table D.6: Admissible index values for A2. 



1,6,0,6,6,0,0,12,0,6,0,0,6,12,0,0,6,0,0,12,0,12,0,0,0,6,0,6, 
12,0,0, 12,0,0,0,0,6, 12,0, 12,0,0,0, 12,0,0,0,0,6, 1 8 



Table D.7: The first 50 coefficients of the Theta series with starting point at zero for the A2 
lattice. 



A subgroup Ts C Aut(A = I^) of order 8 is given by [139] 



/O -1 

1 



Vo 





1 

1 oy 



/O 



1 

Vo 1 



-1 0^ 
-1 

/ 






Vi 




1 
-1 






0/ 



(D.6) 



D.5 

The Da lattice (also known as the Schlafli lattice or checkerboard lattice) consists of 
all points of Z'^ that have even squared norms [22]. A possible generator matrix is 
given by 

"1 1 ■ 

-10 10 
0-101 
-10-1 



M 



(D.7) 



The Gram matrix is given by 



A 



2 -1-1 -r 

-12 

-10 2 

-10 2 



(D.8) 



and a subgroup Fg C Aut(A = D4) of order 8 is given by ( ID. 61 ). See Table ID. iTI for 
a an overview of important constants related to the D4 lattice. 

If a is 7 or a product of primes congruent to 1 (mod 4) then has a 
geometrically-similar and clean sublattice of index N = a? [28]. This is the set 7 
and integer sequence A004613 [125], see also Table ID. 12] 

The first 50 coefficients of the Theta series for are shown in Table ID. 13] 
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-3-2-10123 



Figure D.2: The first 6 non-zero shells of A2 is here shown as large circles ( inch the one at 
the origin). Notice that the number of points lying on each circle agrees with the corresponding 
coefficient of the Theta series. 



Description 


Notation 


Value 


Dimension 


L 


4 


Fundamental volume 


V 


1 


Packing radius 


Pp 


1/2 


Covering radius 


Pc 


1 


Space-filling loss 


Aoss 


1.5329 dB 


Space-filling gain over 




OdB 


Kissing-number 


il 


8 


Minimal squared dislancc 




1 


Dimensionless normalized 
second moment 


G(A) 


1/12 



Table D.8: Relevant constants for the lattice. 
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1,9,25,49,81,121,169,225,289,361,441,529,625,729,841,961, 
1089,1225,1369,1521,1681,1849,2025,2209,2401,2601,2809, 
3025,3249,3481,3721,3969,4225,4489,4761,5041,5329,5625, 
5929,6241,6561,. . . 



Table D.9: Admissible index values for Z^. 



1,8,24,32,24,48,96,64,24,104,144,96,96,112,192,192,24,144, 
312,160,144, 256,288,192,96,248,336,320,192,240,576,256,24, 
384,432,384,312,304,480,448,144,336,768,352,288,624,576, 
384,96,456 



Table D.IO: The first 50 coefficients of the Theta series with starting point at zero for the 
lattice. 



Description 


Notation 


Value 


Dimension 


L 


4 


Fundamental volume 


V 


2 


Packing radius 


Pv 




Covering radius 


Pc 


PpV2 


Space-filling loss 




1.1672 dB 


Space-filling gain over 


-Dgain 


0.3657 dB 


Kissing-number 




24 


Minimal squared distance 


mm 


2 


Dimensionless normalized 
second moment 


G(A) 


0.076603 



Table D.ll: Relevant constants for the D4 lattice. 



1,5,7,13,17,25,29,37,41,53,61,65,73,85,89,97,101,109, 
113,125,137,145,149,157,169,173,181,185,193,197,205, 
221,229,233,241,257,265,269,277,281,289,293,305,313, 
317,325,337,349,353,365,373,377,389,397,401,409,421,. . . 



Table D.12: Admissible index values for D4. 
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1,0,24,0,24,0,96,0,24,0,144,0,96,0,192,0,24,0,312,0,144, 
0,288,0,96,0,336,0, 192,0,576,0,24,0,432,0,3 12,0,480,0, 
144,0,768,0,288,0,576,0,96,0 



Table D.13: The first 50 coefficients of the Theta series with starting point at zero for the Dn 
lattice. 



Appendix 



E 



Proofs 
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Proof of Lemma 12.3.71 Most of the work towards proving the lemma has already been 
done in [21] and [28] and we only need some simple extensions of their results. For 
the proof is trivial, since any odd integer is an admissible index value [28] and 
the product of odd integers yields odd integers. Now let a.b E Z+ be odd integers 
that can be written as the product of primes from a certain set s. It is clear that the 
product ab is also odd an can be written as the product of primes of s. For an 
integer is an admissible index value if it can be written as a product of a set of primes 
which are congruent to 1 (mod 4) and/or congruent to 3 (mod 4) [21, 28]. For A2 an 
integer is an admissible index value if and only if it is a product of primes which are 
congruent to 1 (mod 6) [21, 28] and if m is a product of primes which are congruent 
to 1 (mod 4) then is an an admissible integer for L>4 [28] It follows that the 
lemma holds for the lattices mentioned above. Finally, for and L = 4fc, where 
fc > 1, an integer is an admissible index value if it is odd and can be written on the 
form m^/^ for some integer m [28]. Let a — m^/"^ and h — (m')^/^ we then have 
that ab = TO^/^(m')^/^ = (m")^/^, where m" — mm! is odd and therefore an 
admissible index value for Z^. □ 

Proof of Lemma \2.3.2\ The cyclic submodule Aq = ^oA is closed under multiplica- 
tion by elements of A so for any ^' — ^1^2 ■ • -^K-i G A and any Ao G Aq it is 
true that ^'Aq G Aq which further implies that K-n C Aq since ^'Aq G At^. More- 
over, multiplication is commutative in Z, and S so the order of the set of elements 
Co, • ■ ■ iS,K-i when forming A^ is irrelevant. Thus, K^, C A^ and it is therefore a 
product lattice. □ 

Proof of Lemma \2.3.3\ Since the rings considered are unique factorization rings there 

^'We have excluded the index value obtained for m = 7, since this particular index value cannot be 
written as a product of primes mod 4 but is a special case found in [28] . 



173 



174 



(Appendix E) 



Proofs for Chapter |2] 



must be an element ^' e A such that ^o^' = £,n, where is unique up to muhiplica- 
tion by units of the respective rings. However, a unit u E A belongs to Aut(A) and 
multiplication by u is therefore an isometric operation which takes a lattice to itself. 
It follows that ^0^' G Ao for any G Ao which implies that A^ C Aq = A. Once 
again we invoke the fact that Z, and are multiplicative commutative rings from 
which it is clear that A^ C A,, = A, i = 0, . . . , /I - 1. □ 



Proof of Lemma \2.3.4\ Follows trivially from the fact that Gaussian integers commute 
and Lipschitz integers include Gaussian integers as a special case where the and 
fc*'' elements are both zero. □ 
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In this appendix we present a method to numerically estimate ipL for any L and K. 

F.l Algorithm 

In Chapter|5]we presented closed-form expressions for ipi, for the case of K ^ 3 and 
L = 2 or odd as well as for the asymptotic case of i — > oo. In order to extend these 
results to X > 3 it follows from the proof of Theorem l5.3.2l that we need closed-form 
expressions for the volumes of all the different convex regions that can be obtained by 
K —1 overlapping spheres. With such expressions it should be straightforward to find 
ipL for any K. However, we will take a different approach here. 

Let i> be the volume of the sphere V, which contains the exact number of sublattice 
points required to construct N distinct X-tuples, where the elements of each -tuple 
satisfy 1 1 — 1 1 < r, where r is the radius of V. Notice that V is the expanded sphere. 
Thus, the volume of y is times larger than the lower bound of ( 15.261 ). Now let 
!>' = I'/tp^ denote the volume of a sphere that achieves the lower bound (15.26b so that 
N = (/>'/ i^s)^^^ (at least this is true for large N). But this implies that asymptotically 
as the number of lattice points in V goes to infinity we have 



where, without loss of generality, we can assume that Vs = 1 (simply a matter of 
scaling). For a given r in ( IF.2l i we can numerically estimate N, which then leads to 
an estimate of ip^. To numerically estimate N it follows that we need to find the set 




(F.l) 



which leads to 




(F.2) 
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of lattice points within a sphere V of radius r. For each of these lattice points we 
center another sphere of radius r and find the set of lattice points which are within the 
intersection of the two spheres. This procedure continues K — 1 times. In the end we 
find N by adding the number of lattice points within each intersection, i.e. 

iv = |] 1^ • • • 5^ |A, n v{Xk-2) n • • • n v{Xo)\, (f.3) 

Ai As Ak-2 

where 

Ai = {Ai:AieA,ny(Ao)}, 

A2 = {As : A2 e A« n y(Ai) n V{Xo)}, 

(F.4) 

Ak-2 = {Xk-2 ■■ Xk-2 e A,, n viXK-3) n • • • n f(Ao)}. 

As r gets large the estimate gets better. For example for if = 4, A = and 
r = 10, 20, 50 and 70 then using the algorithm outlined above we find V2 ~ 
1.1672, 1.1736, 1.1757 and 1.1762, respectively. 
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In this appendix we give an example of part of a complete assignment. We let 
A = Z^, K ~ 2 and N — 101 and construct 2-tuples as outlined in Sections [5.3.31 
and 15.5.2] These 2-tuples are then assigned to central lattice points in 14- (0). Since 
N = 101 then (at least theoretically) each sublattice points will be used 101 times. 
Furthermore, for a given sublattice point, say Aq G As, the associated sublat- 
tice points, i.e. the set of sublattice points representing the second coordinate of the 
2-tuples having Aq as first coordinate, wiU be approximately spherically distributed 
around Ao (since V forms a sphere). Fig. |G.l(a)| shows the set of A^ sublattice points 
given by 

{Ai G As : Ai = ai(Ac) and ao(Ac) = (1, -10), A^ G A,}, (G.l) 

which represent the set of second coordinates of the A^ 2-tuples all having Aq = 
(1,-10) as first coordinate. Each 2-tuple is assigned to a central lattice point. This 
assignment is illustrated in Fig. |G.l(b)| Here a dashed line connects a given 2-tuple 
(represented by its second coordinate Ai) with a central lattice point. These A^ assign- 
ments are also shown in Table IG.ll 
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40 
30 
20 
10 

-10 
-20 
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r 6' ^ Ox' 
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' ' / I, \ \ \ \ ^ 



(4 ^' P 1 \ 



b ^ 



-50 




(b) 



50 



Figure G.l: The square marks the sublattice point An — (1, —10) and the small circles illus- 
trate the 101 sublattice points which are associated with Ao. (a) The large circle emphasize that 
the sublattice points are approximately spherically distributed around Aq. (b) The assignments 
are illustrated with dashed lines and the small dots represent central lattice points. 
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Ac e A 


ai{Xc) e As 


Ac e A 


ai(Xc) e As 


\ ^ A 

Ac e A 


a\\\c) e As 


(-27,-6) 


(-50,-5) 


(21,-8) 


/A 1 /C\ 

(41,-6) 


(-3,-32) 


/ C C 1 \ 

(-5,-51) 


(-27,-4) 


/CI c\ 

(-51,5) 


(20,2) 


/on 1 ,1 \ 

(39,14) 


(-10,-32) 


(-25,-53) 


(-26,-14) 


(-49,-15) 


(17,-5) 


(30,3) 


(6,-31) 


(15,-49) 


(-25,-16) 


(-48,-25) 


(16,-9) 


(31,-7) 


(5,-31) 


(5,-50) 


(-23,-20) 


(-47,-35) 


(15,1) 


(29,13) 


(20,-30) 


(35,-47) 


(-23,4) 


(-42,16) 


(14,1 1) 


(27,33) 


(15,-30) 


(25,-48) 


(-23,7) 


(-43,26) 


(14,7) 


(28,23) 


/T^ 

(22,-29) 


f AC A £i\ 

(45,-46) 


(-22,-3) 


(-41,6) 


(14,-12) 


(32,-17) 


(-14,-29) 


(-26,-43) 


(-21,-16) 


(-38,-24) 


(11,-8) 


(21,-8) 


(-16,-29) 


(-36,-44) 


(-20,-7) 


(-40,-4) 


(10,6) 


(18,22) 


(16,-26) 


(34,-37) 


(-19,-12) 


(-39,-14) 


(10,-1) 


(19,12) 


(5,-26) 


(4,-40) 


(-16,4) 


(-32,17) 


(10,-4) 


(20,2) 


(-3,-26) 


(-6,-41) 


(-16,11) 


(-33,27) 


(5,-4) 


(10,1) 


(24,-25) 


(44,-36) 


(-15,-7) 


(-30,-3) 


(3,-1) 


(9,11) 


(-9,-25) 


(-16,-42) 


(-15,-2) 


(-31,7) 


(5,18) 


(6,41) 


(13,-24) 


(24,-38) 


(-14,-12) 


(-29,-13) 


(7,18) 


(16,42) 


(8,-24) 


(14,-39) 


(-13,4) 


(-22,18) 


(-5,17) 


(-14,39) 


/ IT /I \ 

(-13,-24) 


(-27,-33) 


(-13,8) 


(-23,28) 


(-3,17) 


(-4,40) 


(-17,-24) 


(-37,-34) 


(-10,-1) 


(-21,8) 


(-13,16) 


(-24,38) 


(7,-19) 


(13,-29) 


(-9,-9) 


(-19,-12) 


(5,13) 


(7,31) 


(-5,-19) 


(-7,-31) 


(-0,-4) 


(-20,-2) 


/CI o\ 

(-5,12) 


(-13,29) 


/ o 1 n\ 
(-0,-19) 


(-17,-32) 


(-8,6) 


(-12,19) 


(-2,12) 


(-3,30) 


(-15,-19) 


(-28,-23) 


(-5,0) 


(-11,9) 


(9,11) 


(17,32) 


(17,-18) 


(33,-27) 


(28,-4) 


(50,5) 


(-2,7) 


(-2,20) 


(3,-18) 


(3,-30) 


(28,-6) 


(51,-5) 


(5,6) 


(8,21) 


(11,-17) 


(23,-28) 


(28,-19) 


(53,-25) 


(1,2) 


(-1,10) 


(2,-16) 


(2,-20) 


(25,3) 


(49,15) 


(-6,-37) 


(-14,-62) 


(-8,-16) 


(-18,-22) 


(24,-11) 


(52,-15) 


(14,-36) 


(26,-58) 


(14,-14) 


(22,-18) 


(24,-14) 


(42,-16) 


(7,-36) 


(16,-59) 


(-4,-14) 


(-8,-21) 


(22,6) 


(38,24) 


(3,-36) 


(6,-60) 


(8,-13) 


(12,-19) 


(22,-18) 


(43,-26) 


(-4,-34) 


(-4,-61) 


(-4,-11) 


(-9,-11) 


(21,13) 


(37,34) 


(-9,-33) 


(-15,-52) 


(6,-9) 


(11,-9) 


(21,-3) 


(40,4) 


(-17,-33) 


(-35,-54) 


(-1,-8) 


(1,-10) 


(-4,-6) 


(-10,-1) 


(-2,-3) 


(0,0) 







Table G.l: The assignments of the N = 101 2-tuples which all have Ao = (1, —10) as first 
coordinate, i.e. ao(Ac) = (1, —10). 



Appendix X J 

Proofs for Chapter i 



For notational convenience we will in this appendix use the shorter notation 5£ instead 



H.1 Proof of Theorem 5.3.1 



In order to prove Theorem l5.3.1l we need the following results. 
Lemma H.1.1. For 1 < k < A' we have 



Ac, ^ ^ A,; 



Proof. Expanding the sum on the left-hand- side leads to (^) k different terms of the 
form (Ac, Ai), where i e {0, . . . ,K — 1}. There are K distinct A^'s so the number of 



times each A, occur is 



k/K. 



□ 



Lemma H.1.2. For 1 < k < K we have 



E 



Proof. There are (^) distinct ways of adding k out of K elements. Squaring a sum 
of K elements leads to /t squared elements and 2(2) cross products (product of two 
different elements). This gives a total of (^)k squared elements, and 2(^) (2) cross 
products. Now since there are K distinct elements, the number of times each squared 
element occurs is given by 



K-2 K-l 



i=0 j=i+l 



#1 



Aill 



k K 



(H.l) 
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There are (^) distinct cross products, so the number of times each cross product 
occurs is given by 



# 



K\ 2(2) 2k{k- 1) fK 



k) Q K{K-1)\k 



Lemma H.1.3. For ii' > 1 we have 



iK-i)Y,\\x,f-2j2 E(^-^^-) = E E 11^^ -^.11^ 

2=0 1=0 j=i+l 1=0 j=i+l 

Proof. Expanding the right-hand- side of (IH.3I ) yields 

E E iiA.-A,ir-E E (iiA.ip + iiA,ir-2(A„A,)) 

2=0 i—0 

We also have 

K-2 K-1 



(H.2) 

□ 



(H.3) 



(H.4) 



E E (11^^ 

i=0 j=i+l 



K-2 K~2 K-1 

iA,f) = E(^^'-i-*)ii^'ii' + E E ii^^ii' 

i=0 1=0 j=i+l 

K-2 K-1 

i=0 j=l 
K-1 K-1 

i=0 j=0 
K-1 K-1 K-1 

= E(^-i)II^'^II'-E*II^^II' + EjII^^II' 



i=0 



3=0 



K-1 



i=0 

which completes the proof. 

We are now in a position to prove the following result. 
Proposition H.1.1. For 1 < k < K we have 



(H.5) 

□ 



E 



K-1 
K ^ ^ ■' 



J=0 



K-1 



K - K 



i=0 
K-2 K-1 



E E 11^'-^. 

2 = J = 2+l 
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Proof. We have 



K— 1 



i-0 



= ||A.f-2(A. igA,) + i, 



J=0 



Hence, by use of Lemmas lH. 1 . 1 l and lH. 1 .21 we have that 



E 



- K — l 



A'-l 



2(k- 1) 



K-2 K-l 



Kk 



i=0 



E E (^-^^■) 



K(K -1)k ^ ^ 



j=0 



1 



K-2 A-l 



Kk 
K 

K 



i=0 



^ ' i=0 j=i+l 



K-l 



/ 2(/.-l) 
\K{K - 1) 



^ j=0 j=i+l / 



K-l 



A' — K 



K-l 

EiiA^ 

i=0 



K ~K 



K^k{K - 1) 
so that, by Lemma |H.1.3l we finally have that 



K-2 K-l 

2E E (A., A, 

j=0 j=i+l 



)( ^'-rS^- 

^ \ 2 = 



K -K 



K^k{K- 1) 



/f-2 A'-l 



E E l|A^:-A 

i=0 j=i+l 



which completes the proof. 



□ 
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Theorem 153111 Forl< k< K we have 



EE 

Ac 



3=0 




K-1 



i=0 

„ x K-2 K-1 



K^k{K -1) 



■J I 



Proof. Follows trivially from Proposition lH.l.ll 



1=0 j=i+l 



□ 



H.2 Proof of Theorem 5.3.2 



Theorem I5.3.2[ For the case of K ^ 3 and any odd L, the dimensionless expansion 
factor is given by 



where [3^ is given by 



/ X 1/2L / r , 1 \ l/2i 



2L 



r,-l/2L 

Pl ' 



(H.6) 



ra=0 



2 \ 2-^~"( — 1)" \ 2 Jk \ 2 J 



L + 1 
2 

n 



k-. 



=0 \ 2 Ik 



i-iy ( IV ' . 

^ ^ ^ Aj L + n + j 



(H.7) 



Proof. In the following we consider the case of X = 3. For a specific Ao E As 
we need to construct 3-tuples all having Aq as the first coordinate. To do this we 
first center a sphere V of radius r at Ao, see Fig. 15.21 For large N and small i/s 
this sphere contains approximately j>/j^s lattice points from As. Hence, it is possible 
to construct {pjvsY distinct 3-tuples using lattice points inside V . However, the 
maximum distance between Ai and A2 points is greater than the maximum distance 
between Aq and Ai points and also between Aq and A2 points. To avoid this bias 
towards Aq points we only use 3-tuples that satisfy ||Ai — A^ H < r for j = 0, 1, 2. 
However, with this restriction we can no longer form A'^ 3-tuples. In order to make sure 
that exactly A^ 3-tuples can be made we expand V by the factor V'l. It is well known 
that the number of lattice points at exactly squared distance I from c, for any c G is 
given by the coefficients of the Theta series of the lattice A [22]. Theta series depend 
on the lattices and also on c [22]. Instead of working directly with Theta series we 
will, in order to be lattice and displacement independent, consider the L-dimensional 
hollow sphere '£ obtained as "if = ^(c, m) — S'(c, m — 1) and shown in Fig. lH.lt a). 
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Figure H.l: The number of lattice points in the shaded region in (a) given by am ~ VolC^) jvs 
and in (b) it is given by hm = Vol{^1a) jvs- 



The number of lattice points a„i in is given by j'^^'n A| and asymptotically as i/^ — s- 
(and independent of c) 

a™ = VolC/)/i/s = — (m^ - (m - 1)^). (H.8) 

The following construction makes sure that we have || Ai — A2 11 < r. For a specific 
Ai e V^(Ao) n As we center a sphere V at Ai and use only A2 points from V"(Ao) H 
V^(Ai) n As. In Fig. IH.H b) we have shown two overlapping spheres where the first 
one is centered at some Aq and the second one is centered at some Ai e l^(Ao) which 
is at distance m from Aq, i.e. ||Ao — Ai|| — m. Let us by '£ denote the convex region 
obtained as the intersection of the two spheres, i.e. = V(\q) n V^(Ai). Now let hm 
denote the number of lattice points in fl A^. With this we have, asymptotically as 
Vs 0, that 6.,„ is given by 

&,„ = VolC^)/i/s. (H.9) 

It follows that the number T of distinct 3-tuples which satisfy || A,; — Aj || < r is given 
by 

r 

lim T=y a™ 6„. (H.IO) 

ni—l 

We now proceed to find a closed-form expression for the volume of which 
eventually will lead to a simple expression for 6,„. Let 2^1 (•) denote the Hypergeo- 
metric function defined by [115] 

2^l(a,6;c;z) = £^^M|tz^ (H.ll) 

k=o 

where {■)k is the Pochhammer symbol defined as 

a fe = <^ (H.12) 
[a{a+l)---{a + k-l) k > 1. 



186 



(Appendix H) 



Proofs for Chapter |5] 



Lemma H.2.1. The volume of an L-dimensional (L odd) spherical cap V^ap is given 
by 

Vol(F,ap) - ^^r(^-i)/2(2r - m)(^+''>/^ 

1 

L + l 1-L L + 3 2r-m\ ^^'^^^ 
4r J ' 

Proof. This is a special case of what was proven in [143] and we can therefore use the 
same technique with only minor modifications. Let = m/2 and let u be a unit vector 
of M^. Furthermore, let H^.u be the affine hyperplane {z + hu\z G R^, 2: • w = 0} of 
R^ which contains the intersection of two spheres of equal radii r and with centers at 
distance m < r apart, see Fig. lH.2l 




m 



Figure H.2: Two balls in K'^ of equal radii r and distance m apart. 



2^1 



We define the spherical cap as 

'^rXu = {zeB{0,r)\z-u>h}, (H.14) 

and its surface is described by 

yrM,u = {zeSiO,r)\z-u>h}, (H.15) 

where B{0, r) G R^ and 5(0, r) G denote the balf^ respectively the sphere of 
radius r and centered at the origin. 

'-In this proof we redefine the concept of a sphere to be in line with [143]. As such, the tenn sphere 
denotes the surface of a ball, hence, a sphere has no interior This terminology is only needed in this proof. 
Elsewhere we define the sphere to be a solid sphere (i.e. a ball and its surface) as is customary in the lattice 
literature. 
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The sphere H^^u H S^r,h.u has radius x = \fr^^~^T? and it is clear that h = 
\/r'^ — x^. Moreover, any point of ,5^r,h.u which is at distanc^ t from Hh.u is at 
distance [x^ — — 2thy/^ from the real line Mm (i.e. the span of u). Hence, the 
volume 'Vol{'T^r,h,u) of '^r,h,u is given by 

/^r — h 



r-h 

ujL-i {{r~h-t){r + h + t))'^^-^^^^dt (H.16) 



= LOL-i / [a-tyit-pydt, 

Jo 

where a — r — h,P=—r — h and 7 = (L — l)/2. The last integral in ( IH.16I ) can be 
shown to be equal to [69, Eq. 3.196.1] 



£ia - trit - dt = ^-^i-pl,^, (^1, -7; 7 + 2; ^] , (H.17) 



which by use of ( IH.35I ) can be written as 



Q,7+i(_m7 / a 



a 



(3 J 7 + 1 ^ ' " ' 'a-/3 

The volume Vol('^r,/i,u) follows by inserting (IH.I8I 1 in ( IH.16I ). that is 

VolC^,,,,,„) = cc-L-i 1 - -5 7T^2^i 7 + 1, -7; 7 + 2; 



(H.18) 



/?; 7 + 1 V a-^ 

L + 1 ^ ' 

X 2=^1 U/2 + 1/2, 1/2 - i:/2; L/2 + 3/2; j , 

(H.19) 

which completes the proof. □ 



The region consists of two equally sized spherical caps. Inserting ( IH.8I 1 



' By distance we mean the length of the shortest straight line that can be drawn between -ff/j.^ and 
.9"^ h It is clear that this line is perpendicular to 
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and ( |H.9l l into ( IH.lOl i leads tc0 (asymptotically as Vs — > 0) 

r 

^ — ^ ^ ^m^ra 



m=l 



/r> M-r+iw? ^ fL+l l-L L + 3 2r — m 



2 ' 2 ' 4r 



r 

X ^ (to"'' — (to — l)^)m'^m^ 

m— 1 

-L + l 

' ^ ' n=0 ^ ' 

X [ L ^ TO^-l+"+^' + 
\ m=l 

-L + l 

(c) 2cJiLi;i_i ^ /^"^^\ 



-r 



t'2(i + 1) V ^1 



X 



(H.20) 



X f r^+"+J + ^ (r^-i+"+^)^ , 

\L+n+j ^ V 

where (a) follows by use of the binomial series expansion [56, p. 162], i.e. (x + y)'' = 
Sn=o "y"' which in our case leads to 

L + l 
2 / L+l \ 

i2r-m)^ = J2{ ^ (2r)^-"(-l)"TO" (H.21) 



n=0 



^^In this asymptotic analysis we assume tliat all Ai points within a given V is at exact same distance 
from the center of V (i.e. from Aq). The error due to this assumption is neglectable, since any constant 
offset from m will appeal' inside 
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and 



(b) is obtained by once again applying the binomial series expansion, that is 

(to - 1)^ = - Lm^-^ + ^(m^"2), (H.23) 

and (c) follows from the fact that X]to=i ™^ = lTT'"^^^ ^(r^). 

Next we let r ^ cx3 so that the number of hollow spheres inside V goes to infin- 
ity@ From dlOOt we see that, asymptotically as — > and r ^ oo, we have 

^^^c^^ L (H.25) 

where is constant for fixed i and given by ( IH.7I I. 

We are now in a position to find an expression for ipL. Let D be equal to the lower 
bound (15.261 1, i.e. P = Vs\/N and let f be the radius of the sphere having volume D. 
Then ^/'l is given by the ratio of r and f, i.e. ^/^l = r/f, where r is the radius of V . 
Using this in ( IH.25I I leads to 



/ Tus{L + l) 
r = 

\2uJLiOL-lLpL 



1/2L 



Since the radius r of an L-dimensional sphere of volume v is given by 

l/L 



V 



(H.26) 



(H.27) 



we can find V'l by dividing ( IH.26I I by ( IH.27I ), that is 



2u>lu:l-iLPl J J 



r 



Since we need to obtain iV 3-tuples we let T = so that with v = \l Nvg we can 
rewrite (IH.28b as 

/ X 1/2L / r , 1 N 1/2L 

^L^i^] Pl''^'. (H.29) 



\i^L-i) \ 2L 

This completes the proof. □ 



^^We would like to emphasize that this is equivalent to keeping r fixed, say r = 1, and then let the 
number of hollow spheres inside V go to infinity. To see this let _A/ ^ oo and then rewrite as 

awM=VolC/)M = i^((^)"-(!I^)'), l<m<M. (H.24) 

A similar change applies to IH.gt . Hence, the asymptotic expression for T is also valid within a localized 
region of which is a useful property we exploit when proving Proposition l5.4.1l 
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H.3 Proof of Theorem 15.3.3 



Lemma H.3.1. For L — > oo we have 



= 1. (H.30) 



Proof. The volume lol of an L-dimensional unit hypersphere is given hy ujl = 
7r^/2/(L/2)! so we have that 



L^oc\{L/2)l 7r^/2-i/2 



1/2L 



lim .V4L(^(i-i))V2L (H.31) 



= 1. 

Lemma H.3.2. For L ^ oo we have 



pi/'"- \v 



□ 



(H.32) 



Proof. The inner sum in ( 15.281 ) may be well approximated by using that sa j- for 
L c, which leads to 



SG)G)""V..G)'i 

Z V4 



We also have that 



2 ' 2 ' 2 ' 4 



- f3/4^(-i+i)/2 fc! (l/2-L/2)fc 

^ ^„ fc!(3/2 + L/2),^ '/^^ 

= (3/4)(-i+^)/2 

L/2-1/2 



X 

fe=0 
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L/2-1/2 

« (3/4)(-i+^)/2 J2 (l/3)^ (H.34) 

k=0 

where (a) follows from the following hypergeometric transformation [115] 

2^1 (a, b; c; z) = (1 - z)-''2=^i (c - a, b; c; , (H.35) 
where ^ = Finally, it is true that 

n=0 ^ ^ 

Inserting (iHJST l. (|H34l ) and ( |H36l ) into dTTTl ) leads to 

L/2-1/2 

/3l « (3/4)(-i+^)/2 i ^ (^/3)fe^ (JJ 37) 



A:=0 



where since X]feLo(l/3)'^ = 3/2, we get 



lim -1- = lim (4/3)i/4(4/3)-i/4^Li/2i(2/3)i/2i 

L^oo ^V2L L^oo (jj 3g) 

= (4/3)l/^ 

which proves the Lemma. □ 

We are now in a position to prove the following theorem. 
Theorem 15.3.31 For K = 3 and L —> oo the dimensionless expansion factor ip^ 
given by 

/4\^/^ 

i'oo = f 3 j . (H.39) 

Proof. The proof follows trivially by use of Lemma lH.3.1l and Lemma lH.3.2l in ( |H.29t . 

□ 



H.4 Proof of Proposition 15.4.1 



Let Ti — {Xi : Xt — ai{Xc), Ac G V^7r(0)}, i.e. the set of iV^ sublattice points G As 
associated with the N'^ central lattice points within T^7r(0). Furthermore, let T"/ C Ti 
be the set of unique elements of Ti, where |r/| w N. Finally, let 

TJ{X^) - {Xj : Xj = aj(Ae) and A, = a,{X,), X, e V^{0)}, (H.40) 

and let Tj(Xi) C Tj (Xi) be the set of unique elements. That is, Tj (Xi) contains all the 
elements Xj G As which are in the if -tuples that also contains a specific A^. We will 
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also make use of the notation #a to indicate the number of occurrences of a specific 
A,inr,(AO. 

For the pair we have 

||a,(A,)-a,(A,)|p= J2 E ll^^-^.ll'- 
Given Ai G T', we have 



^ ||A,-A,|1V.= E #A,I1A.-A, 



(a) TV 



^ J2 l|A.-A,|pz.. 
a,gt;(a,) 

^ II Ai — x||^ dx 

y(A,) 



N 
N 



— I' 



TV 

(b) 



LG{Sl) 



where (a) follows by assuming (see the discussion below for the case of ii' = 3) 
that #Aj = N/N for all A^ G Tj{\i) and (6) follows since v = iVi^^- Hence, with 
v — Nvs — TpN'^/'^^^^^Vs and Ug = A^i^, we have 

I E l|A,:-A,||2z.,«TV^,V^i^2/L^2/L^2/L(K-l)g(^^) 
A,eT,(A.) 

= v,^IN^+^^'^^''-^K^'^G{Sl), 

which is independent of Ai, so that 

iE E iiA.-A,f-^ E 11^^ -^.11' 

A,eT/ Ajer,(A,) AjGTjCA.) 



In ( IH.41I ) we used the approximation jf^x. w A^/iV without any explanation. For 
the case of if = 2 and as ^ cx) we have that T/ ~ Ti and N — N, hence the 
approximation becomes exact, i.e. = 1. This proves the Proposition for K — 2. 
We will now consider the case of X = 3 and show that asymptotically, as L ^ oo, 
the following approximation becomes exact. 

Y E l|A,;-A,||2«iVz>2/iG(5i). (H.42) 

A, ST, (A.) 
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To prove this we use the same procedure as when deriving closed-form expressions 
for ipL leads to the following asymptotic expression 

r 

I|A, - Aj ||2 = ^a„6„m2, (H.43) 

where we without loss of generality assumed that = and used the fact that we 
can replace ||Aj|p by for the \j points which are at distance m from Xi — 0. It 
follows that we have 



^ V HA- \ II 2 _ o ^Lt^L-l 1 ^, 2L+2 



L 

A, ST,- (A,) 



where 



L + l L-1 

— /L+l\ — 

Since !> = w^r^ = ^j^^/Niys we can rewrite ( IH.44I ) as 

"S^ II \ \ ||2 _ o ^-LC^L-l 1 ^, ~2+2/L 1 

^ \\K-x,\\ -2 ^ + ;^5Tvr 

-2^1+2/Li + /L^ ^ 

(^) 2.^i-l 1 oi~2lLi,j( \fL + l\ 1 



+ 1 \^L-lJ 



1 



where (a) follows by inserting (IH.29I I. Dividing ( IH.46b by ( IH.42I ) leads to 

^VLLG{SL)fiL L Ol' 
Hence, asymptotically as L — > cx) we have that 



(H.44) 



(H.45) 



(H.46) 



(H.47) 



lim^l±^^ = l, (H.48) 

which proves the Proposition. 

Remark H.4.1. Proposition 15.4.11 considered the asymptotic case of L ^ oo. Exact 
distortion expressions for the case of X = 3 and finite L follow by replacing ( |H.41t 
with ( 11143] ). 
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Remark H.4.2. For X > 3 it is very likely that similar equations can be found for 
ipL which can then be used to verify the goodness of the approximations for any K. 
Moreover, in Appendix IH. 51 we show that the rate of growth of ( IH.41l i is unaffected 
if we replace by either minA^ {#Aj } or maxA^ {#Aj } which means that the error 
by using the approximation N/N instead of the true is constant (i.e. it does not 
depend on A^) for fixed K and L. It remains to be shown whether this error term tends 
to zero as L oo for K > 3. However, based on the discussion above we conjecture 
that Proposition 15 .4. 1 1 is true for any K asymptotically as N,L ~^ oo and i^g ^ 0. 
In other words, the side distortion of a if-channel MD-LVQ system can be expressed 
through the normalized second moment of a sphere as the dimension goes to infinity. 



H.5 Proof of Proposition 15.4.2 



Before proving Proposition 15.4.21 we need to lower and upper bound (see Ap- 
pendix IH.4I for an introduction to this notation). As previously mentioned the Xj 
points which are close (in Euclidean sense) to Xi occur more frequently than Xj points 
farther away. To see this observe that the construction of X-tuples can be seen as 
an iterative procedure that first picks a Ao G As fl 14- (0) and then any Ai G A^ is 
picked such that ||Ao — Ai|| < r, hence Ai G As D V'(Ao). The set of Xk-i points 
that can be picked for a particular {K — l)-tuple e.g. (Aq, . . . , Xk-i) is then given by 
{Xk-1 ■■ Xk-1 G AsnV^(Ai<:_2)n- • •ny(Ao)}. Itisclearthat ||Ai- Ajll < r where 
{Xi, Xj) = {ai{Xc),aj{Xc)),'^Xc G A^ and any i, j e {0, . . . , K - 1}. 

Let Tinin(Ai, Xj) denote the minimum number of times the pair (A^, Xj) is used. 
The minimum T^^^ of T'njiji(Aj, Xj) over all pairs (A^, A^) lower bounds Nf N. We 
will now show that Tj^„ is always bounded away from zero. To see this notice that the 
minimum overlap between two spheres of radius r centered at Ao and Ai, respectively, 
is obtained when Aq and Ai are are maximally separated, i.e. when ||Ao — Ai|| = 
r. This is shown by the shaded area in Fig. IH.3l for L ~ 2. For three spheres the 
minimum overlap is again obtained when all pairwise distances are maximized, i.e. 
when ||Ai — Aj|| = r for i,j G {0, 1, 2} and i ^ j. It is clear that the volume of 
the intersection of three spheres is less than that of two spheres, hence the minimum 
number of A2 points is greater than the minimum number of A3 points. However, 
by construction it follows that when centering K spheres at the set of points s — 
{Ao, . . . , Xk^i} — {ao{Xc), . . . , (Ac)} each of the points in s will be in the 
intersection Hs of the K spheres. Since the intersection of an arbitrary collection of 
convex sets leads to a convex set [117], the convex hull ^(s) of s will also be in Os- 
Furthermore, for the example in Fig. IH.3I it can be seen that "if (s) (indicated by the 
equilateral triangle) will not get smaller for K > 3 and this is true in general since 
points are never removed from s as if grows. For L = 3 the regular tetrahedron [25] 
consisting of four points with a pairwise distance of r describes a regular convex 
polytope which lies in Og- In general the regular L-simplex [25] lies in iHs and the 
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Figure H.3: Three spheres of equal radius are here centered at the set of points s — 
{Ao, Ai, A2}. The shaded area describes the intersection of two spheres. The equilateral trian- 
gle describes the convex hull 'if{s) of s. 



volume Vol(L) of a regular L-simplex with side length r is given by [13] 

"-^ IL + I 



Vol(L) = - 



2^ 



clt 



(H.49) 



where cl depends only on L. It follows that the minimum number of iiT-tuples that 
contains a specific (Ai, \j) pair is lower bounded by No\{L)^ / . Since the 
volume i> of y is given by i> = lolt^ we get 



Vol(L) 



K-2 



CL 



K-2 



K-2 



(H.50) 



K-2 



(H.51) 



Also, by construction we have that N < (v/vs)^ ^ and that N = v jvg so an upper 
bound on N j N is given by 

N \iys 

which differs from the lower bound in ( IH.SOI l by a multiplicative constant. 
We are now in a position to prove Proposition l5.4.2l 

Proposition |5A2] For N ^ oo and 2 < K < oo we have 

2 



AoGV,(0) 



(H.52) 



Proof. The numerator describes the distance from a central lattice point to the mean 
vector of its associated i^-tuple. This distance is upper bounded by the covering 
radius of the sublattice A^. The rate of growth of the covering radius is proportional 
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tovl^^ = {Nv)^^^, hence 



E 

A,GV„(0) 



-1 ^ = ^ (^N^N^/^v^/^^ . (H.53) 



Since the approximation N/N used in Proposition 15 .4. 1 1 is sandwiched between the 
lower and upper bounds (i.e. Eqs. ( IH.SOb and ( IH.Slb ) we can write 

K-2 K-1 

E E E ll«.(Ac)-a,(A.)|p 

K-2 K-1 

= E E E ll«.(Ac)-a,(A.)|p 
1=0 j=i+i Xaev^(o) 



(H.54) 



so that, since Xi ~ ^^(Ac), 

K-2 K-l 



E E E l|A.-A,||2 = ^(iV2^2K/L(K-l)^2/L 



A<:eV„(0) i=0 j=i+l 



(H.55) 



Comparing ( |H33T i to (|H35] i we see that (|H32] i grows as ^ (TV^^/^^"!)) -> for 

N oo and A' < oo. □ 



Appendix J 

Proofs for Chapter i 



For notational convenience we will in this appendix use the shorter notations 
and instead of if(-^^«), ^/'^'''^ and ^If '"^ 



1. 1 Proof of Theorem 6.2.1 



To prove Theorem l6.2. ll we need the following results. 

Lemma I.l.l. For \ < k <K and any i e {0, . . . , if — 1} we have 



K-\ 



j=0 

Proof. Since | = {^^D the sum J2fjo^ Pi-^j) contains K{^zl) terms. How- 
ever, the number of distinct terms is |^| = (^) and each individual term occurs k 
times in the sum, since 

Subtracting the terms for j — i proves the lemma. □ 



Lemma 1.1.2. For 1 < k < K and any i, j G {0, . . . , K — 1} we have 

K-l 

Y^p{^.,^,) = np{^,). 

j=0 
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Proof. It is true that = ^^ and since |^,| = (^J^^) and | = (^-|) the 
sum ^Yl!^=^ P{-^i-j ) contains (/f — 1 ) {^~2) + 1 ) terms. However, the number of 
distinct I € ££i terms is — (^Zi ) and each term occurs k times in the sum, since 



Lemma 1.1.3. For 1 < k < X we have 

ZG^ \ iel / \ i=0 I 

Proof. We have that 

E p(^) i E ^) - ^ E ^'(o E 



□ 



I -I A'-l 



A'-l 



K 

\ i=0 I 

where the last equality follows since .2i denotes the set of all /-terms that contains the 
index i. □ 

Lemma 1.1.4. For 1 < k < if we have 

K-2 K-1 K-1 
i=0 j=i+l i=0 

K-2 K-1 
-2E E P{^^)Pm{K\,)■ 

Proof. We have that 

Y E pmpm\\K-\,r^Y. E pmpm{m? + u. 

i=0 j=i+l i=0 j=i+l 



K-2 K-1 

i=0 J=i+1 



Furthermore, it follows that 

K-2 K-1 



J2 j2p(-^^)pi-^MM\' + u,r) 

1=0 j=i+l 
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K~2 K~l K-1 i-1 

i=0 i=l i=0 

1=0 j=i+l i=0 i=0 

Ofori=K-l Qioij=0 

K-l K-1 

1=0 i=o 

K-1 

1=0 

where the last equality follows by use of Lemma H. 1.11 □ 
Lemma 1.1.5. For 1 < k < /v we have 

K-2 K-1 K-1 

J2 E p{^^,M^^-XA\'^i^-^)T.P(^^)\\^^\\' 
i=0 j-i+1 i=0 

K-2 K-1 

-2E E P(-^^.)(A.,A,). 

i=0 j=i+l 

Proof. We have that 

K-2 K-1 K-2 K-1 

E E p(-^*.^)ii^^-^^-ii' = E E ^^("^M)(iiA.ir 

K-2 K-1 

+ ||A,|p)-2E E P(^...)(A.,A,). 

i=0 j=i+l 

Furthermore, it foUows that 

K-2 K-1 

E E p(^M)(iiA.iP + iiA,ir) 

i=0 j=i+l 

K-2 K-1 K-2 K-1 

= E E p(^.,.)iiA.iP + E E 

K-2 K-1 K-lj-1 

= E ii^^ii' E + E E^^("^'.^)ii^. 

i=0 j=i+l 3 = 1 i=0 



A' 



A' 
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K-l K-1 K-l j~l 

j=0 j=i+l 3=0 



K-l 



for i=K-l 

' i-1 K-l 



j=0 j=i+l 



i=0 
K-l 



'K-l 



= Eii^^iiM E^'(-^^^)-^'(^') 



(a) 



i=0 
K-l 



j=0 



K-l 



(^-i)E 



where (a) follows by use of Lemma 11.1.21 
Lemma 1.1.6. For 1 < k < we have 



□ 



Ea, 



K-l 



K-2 K-l 



i=0 i=0 j=i+l 



Proof. The set of all elements I of ^ that contains the index i is denoted by 
Similarly the set of all elements that contains the indices i and j is denoted by ^ij- 
From this we see that 



EfW 



Ea, 



(K-2 K-l 
E11A.1P + 2E E (^'-^0 

K-l K-2 K-l 

= E^^("^')ii^^ii' + 2E E p(-^m)(a.,a,). 

i=0 4=0 j=i+l 



By use of Lemma lT.l.Sl it follows that 



Ea. 



A'-l 



Ep(=s^oiiA.ir + («-i)E^^("^')ii^^ii' 

i=0 i=0 

-EE P(^.,.)l!A.-A,ir 

i=0 j=i+l 
K-l K-2 K-l 

K E p(^.)iiA.iP -EE pi^^M^^ - ^.1 

i=0 i=0 j=i+l 



□ 
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We are now in a position to prove the following result. 
Proposition I.l.l. For 1 < k < K we have 

K-l 



Ac 



Xr 



K-2 K-l 



K-2 E E 



1=0 j=i+l 



||A.-A,|| 



(I.l) 



Proof. Expansion of the norm on the left-hand-side in (11.11 ) leads to 



Y^pil) A. --5: A. =5:^(0 ||A.|r-2(A.,-^A, 



iei 



p{^)\\Xcf - 2 (Ac, I P(^OA. ) + A E ^'(O 



1=0 



■Pi^) 



+ iEp(o 



1=0 
2 



1 



Ea 

E^^ 



K-l 



i=0 



P{^) 



E^' 



E^» 



x-i a:-2 k~i 

Y,p{■^^f\M' + 2Y. E P(^.)P(^.-)(A.,A, 

i=0 

K 



i=0 



i=Q 



i=0 



K~2 K-l 



K-l 



i=0 i=i+l ' i=0 

,R--2 K-l 

Y p{J^i)pm\\^^-^A\' 

i=0 



K-l 



- ^p(^,)M=^)-p(^.))l|A 

i=0 



1=0 
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K-2 K-1 



where (a) follows by use of Lemma ll.l.3l and (b) by use of Lemmas II. 1 .41 and ll. 1 .61 



□ 



Theorem I.l.l. For I < k < K v/e have 

2 



Ac > A, 



Ac 



K-1 



i=0 



K-2 K-1 



Proof. Follows trivially from Proposition ll.l.ll 



(1.2) 



□ 



1.2 Proof of Proposition 16.3.1 



Proposition 16.3.11 For K — 2 and asymptotically as iV; oo.Vi — > as well as 

for K = i and asymptotically as Ni,L oo and Vi 0, we have for any pair of 
sublattices, (A^, Aj), i, j — 0, . . . , K — 1, i ^ j. 



K-1 



2/L{K-l) 



Acsy^Co) "1=0 

Proof. Let — {\i : Xi = ai{\c), Ac S V^(0)}, i.e. the set of N.^ sublattice points 
Xi e hi associated with the N-^ central lattice points within 14^(0). Furthermore, 
let T[ C Ti be the set of unique elements of T^. Since (for large N,-,) all the lattice 
points of Aj which are contained within K-(O) are used in some X-tuples, it follows 
that |T/| « v/vi = N„/Ni. Finally, let Tj{\) = {Aj : A^ = aj{\^) and A, = 
ai(Ac), Ac e V7r(0)} and let T^{Xi) C Tj{Xi) be the set of unique elements. That 
is, Tj{\i) contains all the elements e Aj which are in the X-tuples that also 
contains a specific Ai G A^. We will also make use of the notation to indicate the 
number of occurrences of a specific Xj in Tj{Xi). For example for A' = 2 we have 
#Aj — 1, VAj whereas for iiT > 2 we have > 1. We will show later that using 
the approximation ~ ^i/^j is asymptotically good for K — 2>,L oo and 
Nn OO, Vn. Furthermore, we conjecture this to be the case for A' > 3 as well. 
For sublattice Ai and Aj we have 

J2 ||«.(Ac)-a,(Ac)f = E ll^^-^.ll'- 
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Given G T/, we have 



A,GT,(A,) 



II Aj — a;|p da; 



TV, 



since AT,- = t'/j^j. Hence, with z> = ipLi^Yim^o 



, we have 



^ E l|A,-A,||S-^.-.^M"G(5i)n^™^'^^''" 
which is independent of A^, so that 

^E E 11^- ^.11^ 4 1 E 11^- ^.11^ 



1) 



K-1 



T?l = 



(1.3) 



which completes the first part of the proof. We still need to show that for K — 3 
and L — > oo as well as Nm cx),Vm the approximation w Nt/Nj is good. 
That this is so can be deduced from the proof of Proposition |5.4.1| (the last part where 
isT = 3) by using the fact that v = ipi,^Y[ ^m'^^ ^'^ in order to prove that 



1 ^ ||A,-A,|p=7V,£>2/^G(5z 



(1.4) 



A,GT,(A, 



which shows that ( II.3I I is asymptotically true for K — L oo and Nn oo, Vn. 



□ 



1.3 Proof of Proposition 16.3.2 

Proposition 16.3.21 For Ni ^ oo we have 







?'(-^i)Ai 


2 




-p(-^m)) 


l|A.-A,IP 



0. (1.5) 
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Proof. The numerator describes the distance from a central lattice point to the 
weighted centroid of its associated X-tuple. Let us choose Aq such that Nq < Ni,yi. 
Then, since by construction there is no bias towards any of the sublattices, the 
weighted centroids will be evenly distributed around Aq points. Hence, the distance 
from central lattice points to the centroids can be upper bounded by the covering ra- 
dius of Aq. This is a conservativ^ upper bound but will suffice for the proof. The 
rate of growth of the covering radius is proportional to i'q^^ — {Nqv)^/^ , hence 



E 

Ae6Kr(0) 



By use of Proposition |6.3.1| we havj^ 



(1.6) 



K~2 K-1 



P{^) 

- E E ^^^^"^^^^^ 



z E E E 

K-2 K-1 



L 



j=0 j=i+l 



P{^) 



K-1 



i=0 j=i+l 



-p(^,,,)j ||a,(Ae)-a,(A,)||^ 



m=0 



P[ 



SO that, since A^ = ai{Xc), we get by use of Proposition 16.3.1 



K-2 K-1 



E E E ( >q)^(f^^) -p(^,^.))HA.-A,ip 



Xaev^io) ■i=o j=i+i 



(1.7) 



Comparing (|L6ll to (0) we see that (O grows as 9 (^N^^^ /N^^^'-^ ^ for 
N,, ^ oo. □ 



'*The number of distinct centroids per unit volume is larger than the number of points of Aq per unit 
volume. 

^'The approximation of 'n Proposition |6.3. 1 I does not influence this analysis. To see this we refer 
the reader to Appendix lH.51 

'*In this case we actually lower bound the expression and as such the order operator is in fact Q. Recall 
that we say that /(n) = iff{g(n)) if < f(n) < cig(n) and /(n) = r2(g(n)) if /(n) > cog{n), for 
CO, ci > and some large n. Furthermore, /(n) = ©(g(n)) if cog(n) < /{") < cig(n), cf. [56]. 
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V 



14(0) 




Figure I.l: The complete sphere consisting of the regions si and SS describe V-n (0). The radius 
o/ W(0) is tq. The small bright sphere describe V. When V is centered at Ao points within the 
sphere si of radius r\ it will be completely contained within W(0). 

1.4 Proof of Lemmas 

Proof of Letnma \6.2.2\ For simplicity (and without any loss of generality) we assume 
that VTr{0) forms the shape of a sphere, see Fig. 11.11 The X-tuples are constructed by 
centering a sphere V of volume i> around each Ao G Vjr (0) and taking all combinations 
of lattice points within this region (keeping Ao as first coordinate). From Fig. II. II it 
may be seen that any Ao which is contained in the region denoted £/ will always be 
combined with sublattice points that are also contained in V^{0). On the other hand, 
any Ao which is contained in region ^ will occasionally be combined with points 
outside Vt,-{Q). Therefore, we need to show that the volume Vj^ of si approaches 
the volume of V^(0) as A^^ — > oo or equivalently that the ratio of V^/Vj^ ^ as 
Ni — > oo, where denotes the volume of the region 

Let ujl denote the volume of an L-dimensional unit sphere. Then Vj^ = Wirf 
and Vag ^ i/^ — V^, where i^t^ is the volume of V^(0). The radius ri of si can be 
expressed as the difference between the radius tq of ¥-,^{0) and the radius of V, that is 



Since, = t^Jl^^ = ^^^^ and i> = ip^i^U^^^'^ = i^^vNl'^^ we can 
write Vj/ as 



(1.8) 



( 



) 



l/L 



( 




) 



L 



(1.9) 




The volume of can be expressed through the volume of si as 



(1. 10) 
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so that their ratio is given by 

Clearly, for if > 2 we have 
lim 



/L _ ^l/L{K-l) 



(I.ll) 



-i/L{K-i)y 



which proves the claim. 



(1.12) 



□ 



Proof of Lemma |6. 2. 51 We only need to prove Lemma [6.2. 31 for Aq and Ai. Then by 
symmetry it must hold for any pair. Let denote the set of if-tuples constructed 
by centering V at some Aq G V^(0) n Aq. Hence, s e has Aq as first coordinate 
and the distance between any two elements of s is less than r, the radius of V . We 
will assum^that Sx,, + 0,VAo. 

Similarly, define the set c^Ai 7^ by centering V at some Ai G V7r(0) n Ai. 
Assumj^ all elements of the ii'-tuples are in T/^ (0). Then it must hold that for any s e 
we have s G UAoei^ nAo ■^^a- it also true that for any s' e .S^x^ we have 
s' e UAiey^nAi -^Ai- Hence, we deduce that UAosv^nAo ^-^o = UAisv^nAi ^Ai- 
Furthermore, |14(0) n Ao| = A^./TVq, |=5^AoI = ^o,VAo G 14(0) n Aq and ^a;, n 
^A^' = 0, A^ ^ A^', which impUes that | UAoey.nAo I = □ 



1.5 Proof of Theorem 6.5.1 



Before proving Theorem l6.5.1l we need the following results. 
Lemma 1.5.1. For \ < k <K and any I ^ have 



ft-2 K-1 

= «E 11^.11^ -E E I 

2 = j = 'i + l 



I A,,. - A, 



"This is always the case if r > max^ r(Ai) where r(A,;) is the covering radius of the sublattice. 
The covering radius depends on the lattice and is maximized if is geometrically similar to , in which 
case we have [22] 

r(A.) = iV2vi/^</^. 

Since r = ^^u'^/ N^,^ /ui]/ ^ it follows that in order to make sure that .^Xg the index 
values must satisfy 

Af» < (V2V'L)^'^LA^y^"'', i = 0,...,K-l. (*) 
Throughout this work we therefore require (and implicitly assume) that Q is satisfied. 

'*''This is asymptotically true according to Lemma |6.2.2| since we at this point do not consider the cosets of 
the -tuples. Furthermore, the cosets are invariant to which lattice is used for the construction of iC -tuples 
as long as all elements of the _R'-tuples are within 14- (0). 
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Proof. We can write 



which by use of Lemma lH. 1 .3l leads to 



i6i 



-Eii^^n' + ('^-i)Eii^^n'-E E ii^'.-^'.i 
= «E 11^.11' -E E \\>^h-^uf- 

jei 1=0 j=i+i 



□ 



Lemma 1.5.2. For 1 < k < K and any ^ e ^ we have 

\jei i=o I jei 

K-l K-1 



i=0 



f roo/ 

/ if-i \ K-l 

2 ( E ' E ) = 2 E E P(-^0(A, , A,) 

i=o / je; i=o 

= -EEp(-^»)II^^-^^ii' 

jei i=o 

where by use of Lemma ll.l.ll we obtain 

/ K-l \ K-l 

\ jei i=o I jei i=o 

K-l 

jei i=o 



□ 
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Proposition 1.5.1. For < k < K < 3, Ni ^ oo,i'i ^ and any I G ^ we have 



Ac 6 14(0) 



K-1 



i=0 



m=0 



where 



UJ 



(K,l) 



pi^^K 

-EE pi^M-^j) 



i=0 j=i+l 



where (^) = for k 1. 



Proof. We have that 



1 



E^. 



K-1 



E p(^^)^^ 



i=0 



\je; i=o 



which by use of Lemmas |1.5. l| and |I.5.2| leads to 



K-l 



1 



-2 K-l 



jei 'i=o j=i+i 

+ p{^)K J2 -EE - A, 

i=0 1=0 i=i+l 

jei i=o 

K-l 

jei i=o 
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(1-13) 



K — 2 K. — 1 

i=0 j=i+l ' 

It follows from Proposition |6.3.U ( 11.131 ) and Lemma H. 1.1 1 that we can write 

z E ^E^.-^eW^ 

' ^ jel i=0 i=0 i=i+l i=0 j=i+l 



m=0 



K-2 K-1 



K-l 



-EE pmpmWLi^"''G{SL)N^ n ^"/^^ 

j=0 i=i+l ^ m=0 

This completes the proof. 

Proposition 1.5.2. For any 1 < k < K and Z G ^ we have 



K-l) 



□ 



E 

AeGy„(o) 



iE^.-^E^(^OA. 



K-l 



-1) 



m=0 



Proof. Recall that the sublattice points Ai and \j satisfy ||Ai — Aj|| < r, where r — 
(y jij^Yl^ is the radius of V . Hence, without loss of generality, we let Aj = r and 
A, = 0, which leads to 



E 

Acei4(o) 



i=0 



\ m=0 / 



since £; = Vit^n™=o^' 



□ 
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Proposition 1.5.3. For l<K<K<3,l€ Ni ^ oo and i/^ — > we have 









2 


E 

Acev;r(o) 






- E 



K-l 



Proof. Let A = ^^^^^ X^^^o and A' i X^jei -^r Wenowfollow[28,Eqs. 

(67) - (72)] and obtain the following inequalities: 

||A'-Ac||' = ||A'-A + A-Aef 

= liA' - A|p + ||A - A,f + 2((A' - A), (A - A,)), 
from which we can establish the inequality 

||A' - A||2 + ||A - A,||2 - 2|((A' - A), (A - A,))| < |1A' - Kf 

< ||A' - A||2 + ||A - A,||2 + 2|((A' - A), (A - \,))\. 
Using the Cauchy-Schwartz inequality we get 

IIA' - All^ + ||A - A,||^ - 2|1(A' - A)||||(A - A,)|l < |1A' - A,f 

< ||A' - A|p + ||A - AclP + 2||(A' - A)||||(A- A,)||. 

which can be rewritten as 

2 

< l|A'-A,f 

<||A'-A|pfl + |l--^^r' 



||A'-Af (1-^^ 



1|A'-A|| 



Summing over Ac G V-^iS^) and observing that || A — AdP > 0, we get 

(||A'-Af -2||A'-A||||A-A,||)< W^^'A? 

< (IIA'-Af + |1A-A,|p + 2||A'-A|1|1A-A,|l), 

which can be rewritten as 



E ii^'-^in 1-2- 

< E 11^^ -^'1 

Ac e 14(0) 



EA.ey„(o)ll^'-^llli^-Acir 



(1.14) 
(1.15) 



<| E 11^' -^11^ 

,Acey^(o) 



X 1 + 



EA.ey,(o)l|A'-A|P EA.ey,(o)l|A'-A|p 



(1.16) 
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By use of ( 11.61 ) and Proposition lI.5.2l it is possible to upper bound the numerator of the 
fraction in ( II.14b by 

Y: I|A' - All II A - A.II = ^ ({N,.)y^N..^/^ YL N^^(^-A , 

AceVx(O) V rn=0 ) 

since the covering radius of the fc*'' sublattice is proportional to {NkvY^^ , where TV^ 
is the minimum of A^^, i = 0, . . . , X — 1. 

By use of Proposition |I.5. H it is easily seen that the denominator in ( II.14I ) grows as 

E l|A' - A|p = ^ (v'l'^N-. 'n ' 
Acev„(o) \ "1=0 / 

hence the fraction in ( |I.14t go to zero for Ni ^ cx3. By a similar analysis it is easily 
seen that the fractions in ( II.I6I 1 also go to zero as Ni — > 00. 

Based on the asymptotic behavior of the fractions in ( 11.141 ) and ( 11.161 ) we see that 
(asymptotically as Ni 00) 

E iiA'-Af< iiAc-A'ii^< w^-W, 

Aeey„(o) A,evv(o) A,6y„(o) 

hence 

E ||A.-A'|1^« Y l|A'-Af, 

which completes the proof. □ 

We are now in a position to prove Theorem l6.5.1l 
Theorein l6.5.1l r/ie side distortion £)(^'') due to reception of descriptions {/}, where 
I G ^ for any \ < k < K < is, asymptotically as L, A^^ — > 00 and Vi — > 0, given 
by 

1=0 

where 



i=0 



where (IJ) — for k = 1. 
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Proof. By use of ( 15.161 1 we can write the distortion as 



L 



Dr 



1 1 



E 



-- E 



Acev^(o) 



(1.17) 



where the second approximation follows from that fact that as Ni oo, the distortion 
due to the index assignment is dominating. Furthermore, by use of Propositions 1133] 
and ll.S.ll in (11.171 ) we are able to write 



1 1 



E iEA.-^E^(-^')A' 

m=0 



which completes the proof. 



□ 



Appendix 



J 



Proofs for Chapter H 



This appendix contains proofs of the Lemmas and Theorems presented in Chapter]?] 

J.l Proofs of Lemmas 

Proof of Lemma \7.2.1\ For K — 3, k ~ 1 and i?^ — > cx) we see from ( 14.391 ) that 



1/3 \ -1 



=»(l-p,)-'«(l + 2ft)-'«2-=''-, 

where the approximation follows from the high resolution assumption which implies 
that 2^^" ^ 1. With this, we can write the optimal single-channel distortion of a 
(3, 1) SCEC, which is given by (|437] |. as 

£)(3,1) _ 



<^q + 1 (J.2) 

where the approximation follows since cr^ ^ 1. We now equalize the single-channel 
distortion of three-channel MD-LVQ (or (3,1) MD-LVQ) and (3,1) SCECs (i.e. we set 
(17.11b equal to ( IJ.2b ) so that we can express pq as a function of N' . This leads to 

l + 2p,= (^)^l-p,)- (J.3) 
Using ( |4.37b we rewrite the two-channel distortion of (3,1) SCECs as 

^(3.2) ^ + 

a2(l + p,) + 2 
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(a) 1 



fe' ^^LA^'2-^^= (J.4) 

where (a) is true at high resolution since ^ 1, (6) follows by replacing 
with ( IJ.lb and inserting ( IJ.3l l and (c) is valid for large iV' since iV' ^ 1 implies that 
Pg w —1/2. Similarly, by using (IJ.3l l in (I4.37l i the optimal three-channel distortion 
can be written as 



D 



(3,3) _ 



a2(i + 2p,) + 3 
V2(l + 2p,) 



3 



2 (J.5) 



^X2 

N' 



where the first approximation is valid when cr^ <C 1 and the second follows since 
Pq « —1/2. Comparing ( I J. 4b and ( IJ.5b to ( 17.12b and ( 17.131 ) shows that three-channel 
MD-LVQ reach the achievable rate-distortion region of a (3, 1) SCEC in the quadratic 
Gaussian case at high resolution. □ 

Proof of Lemma [7. 3. 71 Let A^^'' denote the set of epsilon-typical sequences [24] and 
note that yle^-* must have bounded support since, for any L, fxi^o, ■ ■ ■ ,xl-i) > 



/ fx{xo, ■ ■ .,XL-l)dx < 1. 



Let the side quantizers of an MD-LVQ system be SD entropy-constrained lattice vec- 
tor quantizers. An SD lattice vector quantizer designed for an output entropy of, say 
Ri, for the L-dimensional uniform source with bounded support (in fact matched to 
the support of Ac^^) has a finite number of codewords given by 2^^' . The distortion 
performance of a lattice vector quantizer is, under high-resolution assumptions, inde- 
pendent of the source pdf [22,86]. Therefore, using this quantizer for A^^^ instead 
of a truly uniformly distributed source will not affect the distortion performance but 
it might affect the rate. However, since the bounded uniform distribution is entropy 
maximizing it follows that Ri upper bounds the rate of the quantizer. □ 

Proof of Lemma \7.3.2\ The two-channel distortion of a (3, 2) SCEC is given by 

D^'^'^ ^^alil + pg), (J.6) 
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where from ( |4.39l l we see that 

al = 2(1 - p,)-i/3(l + 2p,)-2/32-4«^ (J.7) 

Inserting ( IJ.7l i into (IJ.6I 1 and setting the result equal to ( I7.21l i. i.e. we normalize such 
that the two-channel distortion of (3,2) SCECs is equal to that of (3,2) MD-LVQ. This 
leads to 

(1 + p,){l p.r^'^l + 2p,)-2/3 = ^i^UN'f, (J.8) 
from which we find that 



(J.9) 



It follows that we can write _D(3,3) 
I?(3'^) = ia|(l + 2p,) 

= ^(l-p,)-V3(i + 2p,)V32-«. 

= ^(1 - p,)-V3yT2(i + _ p,)-V6^-2(^,)-l2- 

^i^2-^^\ (J.IO) 

where the approximation follows by inserting p k, —1/2. The proof is now complete 
since (IJ.lOb is identical to (17.24b . □ 



J.2 Proof of Theorem TTJA 



Since (3, 2) MD-LVQ is closely related to (3, 2) SCECs we can to some extent use 
the proof techniques of [111]. However, there are some important differences. We 
cannot rely on random coding arguments since we are not using random codebooks. 
For example where [111] exploit properties of the entropy of subsets, we need to 
show that certain properties hold for all subsets and not just on average. Furthermore, 
we consider the asymmetric case where the individual codebook rates Ri and binning 
rates Rb^i are allowed to be unequal whereas in [1 1 1] the symmetric case was conside- 
red, i.e. only a single codebook rate Rs and a single binning rate i?b was taken into 
account. 

Theorem 17.3.11 Let X e be a source vector constructed by blocking an ar- 
bitrary i.i.d. source with finite differential entropy into sequences of length L. Let 
J C {0, . . . , K — 1} and let Xj denote the set of codewords indexed by J. The set 
of decoding functions is denoted g J : ^j^j ^ R^- Then, under high-resolution 
assumptions, if 

E[piX,gjiXj))]<D^'''J\ yje.Jf, 
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where p{-, ■) is the squared-error distortion measure and for all S ^ J 

>^7^ + ;^log2(|{As|Aj-s}|), (J-H) 
ies tes 

the rate-distortion tuple {Rtfi, ■ ■ ■ , Rb,{K-i), {^^^'"^■'}jej«r) is achievable. 
Proof of Theorem [7. 3. 71 Define the following error events. 

1. (S'o-X does not belong to Ai^\x). 

2. S'l :Thereexists no indices (jo, ••■,ji<--i)suchthat(Ao(jo),---,Ai<-_i(j/<-_i)) = 
a(Ae)forAc = Q(X). 

3. S'2 : Not all channel indices are valid. 

4. : For some k received bin indices there exists another admissible fc-tuple in 
the same bins. 

As usual we have = UiLo^ '^i ^"d the probability of error is bounded from above 
by the union bound, i.e. < J2fja^ P{^i)- 

Bounding P{(S'q): Applying standard arguments for typical sequences it can be 
shown that P{S'o) for L sufficiently large [24]. We may now assume the event 
S'q, i.e. all source vectors belong to the set of typical sequences and hence they are 
approximately uniformly distributed. 

Bounding P{Si): The source vector X is encoded by the central quantizer using 
a nearest neighbor rule. Since any source vector wiU have a closest element (which 
might not be unique) in '^c and by construction all Ac G have an associated if-tuple 
of sublattice points, it follows that P{Si) — for all L. 

Bounding P{S'2): We only have to prove this for one of the channels. Then by 
symmetry it holds for all of them. Furthermore, since the intersection of a finite num- 
ber of sets of probability 1 is 1 it follows that with probability 1 a codeword A^ given 
Ac can be found in some bin. In the following we assume K < 00. Let Ac be the 
codeword associated with X (i.e. X is quantized to Ac), where X g Ai^\x). Let 
A denote the event that Ao(jo) exists in the codebook '^q, i.e. the event [Ao(jo) G 
''^0, Ao(jo) — «o(Ac)]. We then have that 

^(/o(AoOo)) ^^) = p(/o(AoOo)) ^ Mimn 
+piM\oijo))^mpiA). 

where the first term on the right hand side is zero if we make sure that all Ac's are 
assigned a (unique) ii'-tuple. Therefore, we only have to look at the second term as 
was the case in [111]. We must show that G = P[fo{>^aijo)) 7^ ^^1^] ^ 1, i-e. 



1 - G = P[Xo^ + Ao(jo), < 3 < Mo -1\A], 



(J.12) 
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where Mq = £^o2^^'' ° = 2^(-'^o+^o) is the total number of codewords selected for all 
bins from '^^q and Ao^ indicates the j*^ such selected codeword. Since the codewords 
are chosen independently (uniformly) and with replacement they all have the same 
probability of being equal to Ao(jo), so we let j = and rewrite (IJ.12b as 

l-G^[PiXo,^Xo{jo)\A)f'«. 

The size of is 1*^0 1 and all codewords of '^q are equally probable so 



Taking logs and invoking the log-inequalit}0, Eq. ( IJ.13l l can be rewritten as 

which goes to —oo for L cx3 if i?o + 7o > 7; logad'^oD- By use of Lemma 17. 3. II 
we have |Co| = 2^^« so that 1 - G ^ for L ^ 00 if 70 > 0. 

Bounding P{S'3): Assume we receive k bin indices from the encoder We then 
need to show that there is a unique set of codewords (one from each bin) which form 
an admissible fc-tuple. Let J — {io, . . . , ik-i}- Along the lines of [111] we define the 
following error event for any 5 C J : 

4 : 3jI ^ J., Vz e 5, !s{\s{3s)) = /s(As(js)), 
(As(js):'*^j-s(ij-s)) = aj(Ac), Ac e C^, 

i.e. that there exist more than one admissible fc-tuple in the given fc bins. The event 
(03 can be expressed as — / ^'s- probability of the error event S'^ can be 
upper bounded by 

P{S's) < Y[i^^ - l)P[iX1„Xj-sUj-s)) = «7(Ac)], 

for some A^,, where A* is a randomly chosen vector from for i ^ S. Let {As|Aj_5} 
denote the set of admissible fc-tuples that contains Xj-s so that 

-P[(A5,Aj-sUj-s)) = aj(Ac)] < -pf iTjrp- 

lUes F»l 

We are then able to bound P{S'^) by 

\{^s\Xj-s}\ 



]^2^(^'-«^-)|{As|Aj_5}|, 



'The log-inequality is given by log(2:) < 2: — 1, 2: > 0, where log denotes the natural logarithm. 
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which goes to zero if 

>^7. + ^log2(|{As|Aj-s}|). (J.14) 

Finally, the expected distortion is bounded by P{S"^)D,j + P{S')d!a!ix,yj G 
where P{S') for L ^ oo and assuming that the distortion measure is bounded, 
i.e. dmax < oo, proves the theorem]^ □ 



We here make the assumption, as appears to be customary, that the distortion measure is bounded also 
for sources with unbounded support. 



Appendix 



K 



Results of Listening Test 



In this appendix we present the results of the MUSHRA Hstening test described in 
Chapter 
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Figure K.1: MUSHRA test results averaged over all three audio clips for p = 0.1, 0.3 and 
p = 0.5. The seven signals appear in the following order: Hidden ref., 3.5 kHz, 7 kHz, K — 
1,K = 2,K = 3andK = 4. 
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Orig 3.5kHz 7kHz K=1 
(b) p = 0.3 
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(c) p = 0.5 
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Figure K.2: MUSHRA test results for the jazzfragment andp = 0.1, 0.3 and p = 0.5. 
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Orig 3.5kHz 7kHz K=1 
(b) p = 0.3 
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(c) p = 0.5 
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Figure K.3: MUSHRA test results for the speech fragment and p = 0. 1 , 0.3 and p = 0.5. 
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Orig 3.5kHz 7kHz K=1 
(b) p = 0.3 
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Orig 3.5kHz 7kHz K=1 
(c) p = 0.5 
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Figure K.4: MUSHRA test results for the rock fragment and p = 0.1, 0.3 andp = 0.5. 



Samenvatting 



Internetdiensten zoals het voice over Internet-protocol (VoIP) en audio/video stream- 
ing (b.v. video op verzoek en video vergaderen) worden steeds populairder door de 
recente groei van breedbandnetwerken. Dit soort "real-time" diensten vereisen vaak 
een lage verzendtijd, een hoge bandbreedte en een lage pakket-verlies kans om ac- 
ceptabele kwaliteit voor de eindgebruikers te leveren. De heterogene communicatie 
infrastructuur van de huidige pakketgeschakelde netwerken verschaffen echter geen 
gegarandeerde prestaties met betrekking tot bandbreedte of verzendtijd en daarom 
wordt de gewenste kwaliteit over het algemeen niet bereikt. 

Om een bepaalde mate van robuustheid te bereiken op kanalen waarop fouten 
kunnen voorkomen, kan multiple-description (MD) coding toegepast worden. Dit is 
een methode waar de laatste tijd erg veel aandacht aan is besteed. Het MD probleem 
is in wezen een gecombineerd bron-kanaal coderingsprobleem dat gaat over (het met 
verlies) coderen van informatie voor transmissie over een onbetrouwbaar ii'-kanalen 
communicatie systeem. De kanalen kunnen falen, met als resultaat het verlies van 
een pakket en daardoor een verlies van informatie aan de ontvangende kant. Welke 
van de 2^ — 1 niet-triviale deelverzamelingen van de K kanalen falen, wordt bekend 
verondersteld aan de ontvangende kant, maar niet bij de encoder. Het probleem is 
dan een MD schema te ontwerpen dat, voor gegeven kanaal rate (of een gegeven som 
rate), de distorsies minimahseert die een gevolg zijn van reconstruering van de bron, 
gebruik makend van informatie van wiUekeurige deelverzamelingen van de kanalen. 

Hoewel wij ons in dit proefschrift hoofdzakelijk richten op de informatie theo- 
retische aspecten van MD codering, zullen we voor de volledigheid ook laten zien 
hoe het voorgestelde MD coderingsschema kan worden gebruikt om een perceptueel 
robuuste audio coder te construeren, die geschikt is voor b.v. audio- streaming op 
pakketgeschakelde netwerken. 

We richten ons op het MD probleem vanuit een bron-codering standpunt en bek- 
ijken het algemene geval van K pakketten. We maken uitgebreid gebruik van lattice 
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vector kwantisatie (LVQ) theorie, hetgeen een goed instrument blijkt, in de zin dat 
het voorgestelde MD-LVQ schema als brug tussen theorie en praktijk dient. Voor 
asymptotische gevallen van hoge resolutie en grote lattice vector kwantisator dimen- 
sie, tonen wij aan dat de beste bekende informatie theoretische rate-distorsie MD gren- 
zen kunnen worden bereikt, terwijl we, in niet asymptotische gevallen van eindig- 
dimensionale lattice vector kwantisators (maar nog onder hoge resolutie veronder- 
stelUng), praktische MD-LVQ schemas construeren, die vergeUjkbaar met en vaak 
superieur zijn aan bestaande state-of-the-art schemas. 

In het twee-kanaal symmetrische geval is eerder aangetoond dat de zij-representa- 
ties van een MD-LVQ schema zij-distorsies toelaten, die (bij hoge resolutie voorwaar- 
den) identiek zijn aan die van i-dimensionale kwantisators met bolvormige Voronoi 
cellen. In dit geval zeggen wij dat de zij-kwantisators de L-bol grens bereikt. Een 
dergelijk resultaat is niet eerder aangetoond voor het twee-kanaal asymmetrische 
geval. Het voorgestelde MD-LVQ schema is echter in staat de i-bol grens te bereiken, 
bij hoge resolutie voorwaarden, voor zowel het synometrische geval als het asym- 
metrische geval. 

Het voorgestelde MD-LVQ schema schijnt een van de eerste schemas in de litera- 
tuur te zijn die het grootst bekende hoge resolutie drie-kanaal MD gebied in het 
kwadratische Gaussische geval bereikt. Hoewel de optimaliteit alleen voor K < 3 
wordt bewezen, nemen we aan dat het optimaal is voor willekeurige K representaties. 

We laten gesloten-vorm uitdrukkingen zien voor de rate en distorsie prestaties voor 
algemene gladde stationaire bronnen en een kwadratische-fout distorsie criterium en 
voor hoge resolutie voorwaarden (ook voor eindig-dimensionale lattice vector kwanti- 
sators). Er wordt aangetoond dat de zij-distorsies in het drie-kanaal geval kan worden 
uitgedruktin het dimensieloze, genormaUseerde, tweede moment van een L-bol, onaf- 
hankelijk van het type lattice dat wordt gebruikt voor de zij-kwantisators. Dit komt 
overeen met eerdere resultaten voor het geval van twee representaties. 

Het rate verhes wanneer eindig-dimensionale lattice vector kwantisators gebruikt 
worden is onafhankelijk van het lattice en wordt gegeven door het rate verUes van een 
L-bol en een bijkomende term die de ratio van twee dimensieloze expansie factoren 
beschrijft. Er wordt aangetoond dat het totale rate verlies superieur is aan bestaande 
drie-kanaal schemas. Dit resultaat lijkt te gelden voor elk aantal representaties. 
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Glossary of Symbols and Terms 



Symbol 


Description 




-L-dimensional Euclidean space (real field) 




Z/-dimensional complex field 


I? 


Z/-dimensional set of aU rational integers 




Gaussian integers 




Algebraic integers 


S 


Eisenstein integers 




Lipschitz integers 


^\ 


Hurwitzian integers 


x" 


Hermitian transposition (conjugate transposition) 




Quatemionic transposition (Quatemionic conjugate transposition) 




^-module (^-lattice) 


m\ 


Vector norm with respect to underlying field 


{X,X} 


Inner product 



Table K.1: Algebra-related symbols. 
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Glossary of Symbols and Terms 



oymool 


Description 


A 


Scalar random process or L-dimensional random vector (X € K^) 


X 


i-dimensional vector (realization of X) 


fx 


Distribution of X 


X 


Reconstruction of X 




Alphabet of X (usually JT = M^) 




Alphabet of 1 (usually £ C K^) 


p 


Fidelity criterion (usually squared-error) 


R{D) 


Rate-distortion function 


D(R) 


Distortion-rate function 


/(•;•) 


Mutual information 


K-) 


Differential entropy 


K-) 


Differential entropy rate 


H{-) 


Discrete entropy 


E 


Statistical expectation operator 


RsLR 


Shannon lower bound 


-Rloss 


Rate loss 


-"-red 


Rate redundancy 


^Loss 


Space-filling loss 




Variance of X 


Px 


Entropy power 


Q{x) 


Quantization of X 



Table K.2: Source-coding related symbols. 
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Symbol Description 

Ac Central lattice (central quantizer) 

As Sublattice Ag C Ac (side quantizer in symmetric case) 

A, Sublattice A^ C Ac (side quantizer in asymmetric case) 

Ayr Product lattice A^r C Aj or A^ c A^ 

Ac / A^ Quotient lattice 

Vc Voronoi cell of Ac 

V Voronoi cell of Ag or A^ 

v Volume of Voronoi cell of Ac 

Ug Volume of Voronoi cell of Ag 

Ui Volume of Voronoi cell of Aj 
Volume of Voronoi cell of A^r 

A'' Index value of sublattice A^ (N = | Ac/A^ |) 

Ni Index value of sublattice Ag (Ni = |Ac/Ai|) 

A^,r Index value of product lattice A^ (iV^ = |Ac/A^|) 

TV' Nesting ratio of A., (index per dimension) 

G(A) Dimensionless normalized second moment of A 

G{Sl) Dimensionless normalized second moment of i-sphere 

Q Basis vector (lattice generator vector) 

M Lattice generator matrix 

A Gram matrix 

Tm Multiplicative group of automorphisms of order m 

Ac/A^/Fto Set of orbit representatives 

Scalar lattice (uniform lattice) 

Z'^ Square lattice 

Z^ Hypercubic lattice 

A2 Hexagonal two-dimensional lattice 

D4 Four dimensional (checker board) lattice 

^Ac Sublattice of Ac (cyclic right submodule) 

Ac^ Sublattice of Ac (cyclic left submodule) 

^(A) Kissing number of A 



Table K.3: Lattice-related symbols. 
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Glossary of Symbols and Terms 



Symbol Description 



a Index assignment map (q;(Ac) = (Aq, . . . , \k-\)) 

" ^ Inverse index assignment map 

Component function (Ai = a, (Ac)) 
Number of descriptions 
Number of received descriptions 
L-dimensional sphere 
Volume of V 

Number of lattice points of Aj within V 
Dimensionless expansion factor 
Volume of unit L- sphere 
Description rate [bit/dim.] in symmetric setup 
Description rate [bit/dim.] of i*"^ description 
Rate of central quantizer 
Sum rate {Rt = ^Ri) 
Side distortion of i^^ description 
Central distortion 

Distortion due to reconstructing using k descriptions out 

oiK 

Cost functional 
Packet-loss probabiUty 

Index set describing all distinct K-tuples out of the set 

{0,...,X-1} 

Index set describing all distinct K-tuples out of the set 
{0, . . . , -fC — 1}, which contains the index i 
Index set describing all distinct K-tuples out of the set 

{0, . . . , -fi' — 1}, which contains the pair of indices (i, j) 
Probability of the set ^^^f '''^ 

Expected distortion when receiving k out of K descrip- 
tions based on the packet-loss probability 
Distortion due to reconstructing using the subset of de- 
scriptions / C {0, . . . , -ff — 1} 
MD-LVQ Multiple-description lattice vector quantization 
SCEC Source-channel erasure code 
SPSD Sum of pairwise squared distances 
WSPSD Weighted sum of pairwise squared distances 



a 
K 

K 
V 

v 
Ni 

Rg 

Ri 

Rc 
Rt 
Di 
Dc 

J(K) 
P 

r,{K,K) 

'-'a 



Table K.4: MD-LVQ related symbols and terms. 
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